Trace number 3743813

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver? (TO) 1802.22 1802.54

General information on the benchmark

Name/OPT-BIGINT-LIN/sroussel/NG/
normalized-NG.dot_unif_10null0mast_rapportOE2.0_TMAX60_K192_cod1.opb
MD5SUM9306639b4fe31ed54dc3203c6f1fc129
Bench CategoryOPT-BIGINT-LIN (optimisation, big integers, linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark-4876
Best CPU time to get the best result obtained on this benchmark1800.35
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables399113
Total number of constraints400980
Number of constraints which are clauses399002
Number of constraints which are cardinality constraints (but not clauses)1977
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint397089
Number of terms in the objective function 306555
Biggest coefficient in the objective function 999
Number of bits for the biggest coefficient in the objective function 10
Sum of the numbers in the objective function 148323151
Number of bits of the sum of numbers in the objective function 28
Biggest number in a constraint 999
Number of bits of the biggest number in a constraint 10
Biggest sum of numbers in a constraint 148323151
Number of bits of the biggest sum of numbers28
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c SCIP version 1.2.1.3 [precision: 8 byte] [memory: block] [mode: optimized] [LP solver: SoPlex 1.4.2]
0.00/0.00	c Copyright (c) 2002-2010 Konrad-Zuse-Zentrum fuer Informationstechnik Berlin (ZIB)
0.00/0.00	c 
0.00/0.00	c user parameter file <scip.set> not found - using default parameters
0.00/0.00	c reading problem <HOME/instance-3743813-1338794734.opb>
4.89/4.99	c original problem has 399113 variables (399113 bin, 0 int, 0 impl, 0 cont) and 400980 constraints
4.89/4.99	c problem read
4.89/5.00	c presolving settings loaded
6.89/6.90	c presolving:
7.99/8.04	c (round 1) 33998 del vars, 34176 del conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 0 upgd conss, 62458 impls, 1799 clqs
9.99/10.01	c (round 2) 33998 del vars, 34370 del conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 366416 upgd conss, 62458 impls, 1799 clqs
10.58/10.61	c (round 3) 33998 del vars, 34370 del conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 366610 upgd conss, 62458 impls, 1799 clqs
700.79/700.94	c (round 4) 34600 del vars, 34370 del conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 366610 upgd conss, 11494982 impls, 1799 clqs
701.18/701.31	c (round 5) 34600 del vars, 34972 del conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 366610 upgd conss, 11498540 impls, 1799 clqs
712.68/712.87	c (round 6) 36382 del vars, 34972 del conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 366610 upgd conss, 11533330 impls, 1799 clqs
713.18/713.36	c (round 7) 36382 del vars, 36754 del conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 366610 upgd conss, 11533342 impls, 1799 clqs
1087.12/1087.36	c (round 8) 36978 del vars, 36754 del conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 366610 upgd conss, 17647768 impls, 1799 clqs
1087.52/1087.77	c (round 9) 36978 del vars, 37347 del conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 366610 upgd conss, 17647776 impls, 1799 clqs
1800.08/1800.40	c pressed CTRL-C 1 times (5 times for forcing termination)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3743813-1338794734/watcher-3743813-1338794734 -o /tmp/evaluation-result-3743813-1338794734/solver-3743813-1338794734 -C 1800 -W 1900 -M 15500 HOME/pbscip.linux.x86_64.gnu.opt.spx -f HOME/instance-3743813-1338794734.opb -t 1800 -m 15500 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.04 2.01 2.00 3/176 5653
/proc/meminfo: memFree=31142340/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=7916 CPUtime=0 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 1251 0 0 0 0 0 0 0 20 0 1 0 345602052 8105984 1198 33554432000 4194304 8395647 140735767901408 140737126789688 7385025 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 1979 1198 370 1026 0 949 0

[startup+0.026658 s]
/proc/loadavg: 2.04 2.01 2.00 3/176 5653
/proc/meminfo: memFree=31142340/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=19660 CPUtime=0.01 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 4174 0 0 0 1 0 0 0 20 0 1 0 345602052 20131840 4115 33554432000 4194304 8395647 140735767901408 140737126789688 5757902 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 4915 4115 387 1026 0 3885 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 19660

[startup+0.100284 s]
/proc/loadavg: 2.04 2.01 2.00 3/176 5653
/proc/meminfo: memFree=31142340/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=50492 CPUtime=0.09 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 11821 0 0 0 8 1 0 0 20 0 1 0 345602052 51703808 11762 33554432000 4194304 8395647 140735767901408 140737126789688 7370173 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 12623 11762 387 1026 0 11593 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 50492

[startup+0.300285 s]
/proc/loadavg: 2.04 2.01 2.00 3/176 5653
/proc/meminfo: memFree=31142340/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=134932 CPUtime=0.29 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 32432 0 0 0 25 4 0 0 20 0 1 0 345602052 138170368 32373 33554432000 4194304 8395647 140735767901408 140735767766616 4730428 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 33733 32376 387 1026 0 32703 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 134932

[startup+0.700221 s]
/proc/loadavg: 2.04 2.01 2.00 3/176 5653
/proc/meminfo: memFree=31142340/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=252220 CPUtime=0.69 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 61435 0 0 0 61 8 0 0 20 0 1 0 345602052 258273280 61376 33554432000 4194304 8395647 140735767901408 140735767766616 7384814 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 63055 61376 394 1026 0 62025 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 252220

[startup+1.50027 s]
/proc/loadavg: 2.04 2.01 2.00 3/177 5654
/proc/meminfo: memFree=30764364/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=311248 CPUtime=1.49 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 76134 0 0 0 138 11 0 0 20 0 1 0 345602052 318717952 76075 33554432000 4194304 8395647 140735767901408 140735767766616 4502824 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 77812 76075 394 1026 0 76782 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 311248

[startup+3.10029 s]
/proc/loadavg: 2.04 2.01 2.00 3/177 5654
/proc/meminfo: memFree=30662808/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=386772 CPUtime=3.09 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 94968 0 0 0 295 14 0 0 20 0 1 0 345602052 396054528 94909 33554432000 4194304 8395647 140735767901408 140735767766616 7430272 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 96693 94909 394 1026 0 95663 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 386772

[startup+6.30032 s]
/proc/loadavg: 2.04 2.01 2.00 3/177 5654
/proc/meminfo: memFree=29399360/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1086772 CPUtime=6.29 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 268497 0 0 0 592 37 0 0 20 0 1 0 345602052 1112854528 270128 33554432000 4194304 8395647 140735767901408 140735767898744 4602326 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 271693 270128 431 1026 0 270663 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 1086772

[startup+12.7003 s]
/proc/loadavg: 2.03 2.01 2.00 3/177 5654
/proc/meminfo: memFree=28275828/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1599240 CPUtime=12.68 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 386124 0 0 0 1216 52 0 0 20 0 1 0 345602052 1637621760 397671 33554432000 4194304 8395647 140735767901408 140735767897208 4911325 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 399810 397671 502 1026 0 398780 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 1599240

[startup+25.5003 s]
/proc/loadavg: 2.03 2.01 2.00 3/177 5654
/proc/meminfo: memFree=28275760/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1599240 CPUtime=25.49 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 386129 0 0 0 2497 52 0 0 20 0 1 0 345602052 1637621760 397676 33554432000 4194304 8395647 140735767901408 140735767897208 4911325 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 399810 397676 502 1026 0 398780 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 1599240

[startup+51.1003 s]
/proc/loadavg: 2.07 2.02 2.00 3/177 5654
/proc/meminfo: memFree=28275588/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1599240 CPUtime=51.09 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 386139 0 0 0 5056 53 0 0 20 0 1 0 345602052 1637621760 397686 33554432000 4194304 8395647 140735767901408 140735767897208 4911325 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 399810 397686 502 1026 0 398780 0
Current children cumulated CPU time (s) 51.09
Current children cumulated vsize (KiB) 1599240

[startup+102.3 s]
/proc/loadavg: 2.03 2.02 2.00 3/177 5655
/proc/meminfo: memFree=28275136/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1599240 CPUtime=102.28 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 386140 0 0 0 10175 53 0 0 20 0 1 0 345602052 1637621760 397687 33554432000 4194304 8395647 140735767901408 140735767897208 4911325 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 399810 397687 502 1026 0 398780 0
Current children cumulated CPU time (s) 102.28
Current children cumulated vsize (KiB) 1599240

[startup+162.3 s]
/proc/loadavg: 2.06 2.03 2.00 3/177 5655
/proc/meminfo: memFree=28209320/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1644336 CPUtime=162.26 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 396898 0 0 0 16172 54 0 0 20 0 1 0 345602052 1683800064 408445 33554432000 4194304 8395647 140735767901408 140735767897480 4911325 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 411084 408445 502 1026 0 410054 0
Current children cumulated CPU time (s) 162.26
Current children cumulated vsize (KiB) 1644336

[startup+222.3 s]
/proc/loadavg: 2.02 2.02 2.00 3/177 5655
/proc/meminfo: memFree=28121556/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1695572 CPUtime=222.25 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 409477 0 0 0 22170 55 0 0 20 0 1 0 345602052 1736265728 421024 33554432000 4194304 8395647 140735767901408 140735767897560 4911325 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 423893 421024 502 1026 0 422863 0
Current children cumulated CPU time (s) 222.25
Current children cumulated vsize (KiB) 1695572

[startup+282.3 s]
/proc/loadavg: 2.04 2.03 2.00 3/178 5658
/proc/meminfo: memFree=28038248/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1743132 CPUtime=282.25 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 420551 0 0 0 28168 57 0 0 20 0 1 0 345602052 1784967168 432098 33554432000 4194304 8395647 140735767901408 140735767897480 5210338 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 435783 432098 502 1026 0 434753 0
Current children cumulated CPU time (s) 282.25
Current children cumulated vsize (KiB) 1743132

[startup+342.3 s]
/proc/loadavg: 2.01 2.02 2.00 3/177 5658
/proc/meminfo: memFree=27958180/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1784612 CPUtime=342.25 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 430619 0 0 0 34167 58 0 0 20 0 1 0 345602052 1827442688 442166 33554432000 4194304 8395647 140735767901408 140735767897480 4736975 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 446153 442166 502 1026 0 445123 0
Current children cumulated CPU time (s) 342.25
Current children cumulated vsize (KiB) 1784612

[startup+402.3 s]
/proc/loadavg: 2.00 2.02 2.00 3/177 5659
/proc/meminfo: memFree=27916792/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1804008 CPUtime=402.23 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 435559 0 0 0 40164 59 0 0 20 0 1 0 345602052 1847304192 447106 33554432000 4194304 8395647 140735767901408 140735767897480 4911325 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 451002 447106 502 1026 0 449972 0
Current children cumulated CPU time (s) 402.23
Current children cumulated vsize (KiB) 1804008

[startup+462.307 s]
/proc/loadavg: 2.00 2.01 2.00 3/177 5659
/proc/meminfo: memFree=27870448/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1831216 CPUtime=462.23 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 442560 0 0 0 46163 60 0 0 20 0 1 0 345602052 1875165184 454107 33554432000 4194304 8395647 140735767901408 140735767897480 4911325 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 457804 454107 502 1026 0 456774 0
Current children cumulated CPU time (s) 462.23
Current children cumulated vsize (KiB) 1831216

[startup+522.3 s]
/proc/loadavg: 2.00 2.01 2.00 3/177 5659
/proc/meminfo: memFree=27820132/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1862732 CPUtime=522.21 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 449970 0 0 0 52160 61 0 0 20 0 1 0 345602052 1907437568 461517 33554432000 4194304 8395647 140735767901408 140735767897480 6080705 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 465683 461517 502 1026 0 464653 0
Current children cumulated CPU time (s) 522.21
Current children cumulated vsize (KiB) 1862732

[startup+582.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5659
/proc/meminfo: memFree=27768080/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=1892100 CPUtime=582.2 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 456955 0 0 0 58158 62 0 0 20 0 1 0 345602052 1937510400 468502 33554432000 4194304 8395647 140735767901408 140735767897480 4911325 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 473025 468502 502 1026 0 471995 0
Current children cumulated CPU time (s) 582.2
Current children cumulated vsize (KiB) 1892100

[startup+642.3 s]

################
# More data... #
################

/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 513912 0 0 0 100144 70 0 0 20 0 1 0 345602052 2203942912 531591 33554432000 4194304 8395647 140735767901408 140735767897480 4911325 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 538072 531591 502 1026 0 537042 0
Current children cumulated CPU time (s) 1002.14
Current children cumulated vsize (KiB) 2152288

[startup+1062.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5664
/proc/meminfo: memFree=27140892/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=2266072 CPUtime=1062.13 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 525730 0 0 0 106140 73 0 0 20 0 1 0 345602052 2320457728 558739 33554432000 4194304 8395647 140735767901408 140735767897480 6080665 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 566518 558739 502 1026 0 565488 0
Current children cumulated CPU time (s) 1062.13
Current children cumulated vsize (KiB) 2266072

[startup+1122.31 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5664
/proc/meminfo: memFree=26785616/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=2576192 CPUtime=1122.13 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 594685 0 0 0 112126 87 0 0 20 0 1 0 345602052 2638020608 633900 33554432000 4194304 8395647 140735767901408 140735767897272 6080688 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 644048 633900 502 1026 0 643018 0
Current children cumulated CPU time (s) 1122.13
Current children cumulated vsize (KiB) 2576192

[startup+1182.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5664
/proc/meminfo: memFree=26576472/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=2719032 CPUtime=1182.11 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 629976 0 0 0 118117 94 0 0 20 0 1 0 345602052 2784288768 669187 33554432000 4194304 8395647 140735767901408 140735767897272 6080653 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 679758 669187 502 1026 0 678728 0
Current children cumulated CPU time (s) 1182.11
Current children cumulated vsize (KiB) 2719032

[startup+1242.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 5665
/proc/meminfo: memFree=26237452/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=2909884 CPUtime=1242.09 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 674871 0 0 0 124107 102 0 0 20 0 1 0 345602052 2979721216 714082 33554432000 4194304 8395647 140735767901408 140735767897272 6080661 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 727471 714082 502 1026 0 726441 0
Current children cumulated CPU time (s) 1242.09
Current children cumulated vsize (KiB) 2909884

[startup+1302.3 s]
/proc/loadavg: 2.04 2.01 2.00 3/177 5666
/proc/meminfo: memFree=26003996/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=2922172 CPUtime=1302.08 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 679783 0 0 0 130104 104 0 0 20 0 1 0 345602052 2992304128 718989 33554432000 4194304 8395647 140735767901408 140735767897272 6080661 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 730543 718989 502 1026 0 729513 0
Current children cumulated CPU time (s) 1302.08
Current children cumulated vsize (KiB) 2922172

[startup+1362.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 5666
/proc/meminfo: memFree=25821992/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=3039620 CPUtime=1362.07 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 706450 0 0 0 136097 110 0 0 20 0 1 0 345602052 3112570880 745654 33554432000 4194304 8395647 140735767901408 140735767897272 6080701 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 759905 745654 502 1026 0 758875 0
Current children cumulated CPU time (s) 1362.07
Current children cumulated vsize (KiB) 3039620

[startup+1422.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5666
/proc/meminfo: memFree=25713644/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=3051908 CPUtime=1422.06 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 711243 0 0 0 142094 112 0 0 20 0 1 0 345602052 3125153792 750447 33554432000 4194304 8395647 140735767901408 140735767897272 6080661 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 762977 750447 502 1026 0 761947 0
Current children cumulated CPU time (s) 1422.06
Current children cumulated vsize (KiB) 3051908

[startup+1482.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 5669
/proc/meminfo: memFree=25567344/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=3191288 CPUtime=1482.04 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 743222 0 0 0 148087 117 0 0 20 0 1 0 345602052 3267878912 782677 33554432000 4194304 8395647 140735767901408 140735767897272 6080701 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 797822 782677 502 1026 0 796792 0
Current children cumulated CPU time (s) 1482.04
Current children cumulated vsize (KiB) 3191288

[startup+1542.3 s]
/proc/loadavg: 2.08 2.02 2.01 3/177 5669
/proc/meminfo: memFree=25463904/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=3199480 CPUtime=1542.03 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 746513 0 0 0 154084 119 0 0 20 0 1 0 345602052 3276267520 785982 33554432000 4194304 8395647 140735767901408 140735767897272 6080679 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 799870 785982 502 1026 0 798840 0
Current children cumulated CPU time (s) 1542.03
Current children cumulated vsize (KiB) 3199480

[startup+1602.3 s]
/proc/loadavg: 2.03 2.01 2.00 3/176 5669
/proc/meminfo: memFree=25438584/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=3206648 CPUtime=1602.01 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 749553 0 0 0 160081 120 0 0 20 0 1 0 345602052 3283607552 789020 33554432000 4194304 8395647 140735767901408 140735767897272 6080688 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 801662 789020 502 1026 0 800632 0
Current children cumulated CPU time (s) 1602.01
Current children cumulated vsize (KiB) 3206648

[startup+1662.3 s]
/proc/loadavg: 2.01 2.00 2.00 3/177 5670
/proc/meminfo: memFree=25318324/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=3214836 CPUtime=1662.01 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 753002 0 0 0 166079 122 0 0 20 0 1 0 345602052 3291992064 792469 33554432000 4194304 8395647 140735767901408 140735767897272 6080688 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 803709 792469 502 1026 0 802679 0
Current children cumulated CPU time (s) 1662.01
Current children cumulated vsize (KiB) 3214836

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5670
/proc/meminfo: memFree=25159012/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=3372680 CPUtime=1721.99 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 787335 0 0 0 172072 127 0 0 20 0 1 0 345602052 3453624320 826802 33554432000 4194304 8395647 140735767901408 140735767897272 6080665 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 843170 826802 502 1026 0 842140 0
Current children cumulated CPU time (s) 1721.99
Current children cumulated vsize (KiB) 3372680

[startup+1782.31 s]
/proc/loadavg: 2.04 2.01 2.00 3/177 5670
/proc/meminfo: memFree=25132664/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=3379844 CPUtime=1781.99 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 790442 0 0 0 178070 129 0 0 20 0 1 0 345602052 3460960256 830231 33554432000 4194304 8395647 140735767901408 140735767897272 6080675 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 844961 830231 502 1026 0 843931 0
Current children cumulated CPU time (s) 1781.99
Current children cumulated vsize (KiB) 3379844



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.4 s]
/proc/loadavg: 2.03 2.01 2.00 3/178 5671
/proc/meminfo: memFree=25018916/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=3382916 CPUtime=1800.08 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 791550 0 0 0 179879 129 0 0 20 0 1 0 345602052 3464105984 831361 33554432000 4194304 8395647 140735767901408 140735767897272 6080692 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 845729 831361 502 1026 0 844699 0
Current children cumulated CPU time (s) 1800.08
Current children cumulated vsize (KiB) 3382916

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -5653

Solver just ended. Dumping a history of the last processes samples

[startup+1801.7 s]
/proc/loadavg: 2.03 2.01 2.00 3/179 5672
/proc/meminfo: memFree=25017800/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=3382916 CPUtime=1801.38 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 791582 0 0 0 180008 130 0 0 20 0 1 0 345602052 3464105984 831393 33554432000 4194304 8395647 140735767901408 140735767896704 6080661 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 845729 831393 502 1026 0 844699 0
Current children cumulated CPU time (s) 1801.38
Current children cumulated vsize (KiB) 3382916

[startup+1802.11 s]
/proc/loadavg: 2.03 2.01 2.00 3/175 5700
/proc/meminfo: memFree=28090712/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=3382916 CPUtime=1801.78 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4202496 791589 0 0 0 180048 130 0 0 20 0 1 0 345602052 3464105984 831400 33554432000 4194304 8395647 140735767901408 140735767896704 6080692 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 845729 831400 502 1026 0 844699 0
Current children cumulated CPU time (s) 1801.78
Current children cumulated vsize (KiB) 3382916

[startup+1802.5 s]
/proc/loadavg: 2.03 2.01 2.00 3/175 5700
/proc/meminfo: memFree=28090712/32873844 swapFree=7116/7116
[pid=5653] ppid=5651 vsize=0 CPUtime=1802.17 cores=0,2,4,6
/proc/5653/stat : 5653 (pbscip.linux.x8) R 5651 5653 4984 0 -1 4203524 791599 0 0 0 180078 139 0 0 20 0 1 0 345602052 0 0 33554432000 0 0 0 0 0 0 0 4096 16384 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/5653/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 1802.17
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1802.54
CPU time (s): 1802.22
CPU user time (s): 1800.79
CPU system time (s): 1.43578
CPU usage (%): 99.9826
Max. virtual memory (cumulated for all children) (KiB): 3382916

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1800.79
system time used= 1.43578
maximum resident set size= 3325640
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 791599
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 40
involuntary context switches= 2032

runsolver used 3.36849 second user time and 8.94664 second system time

The end

Launcher Data 

Begin job on node103 at 2012-06-04 09:25:35
IDJOB=3743813
IDBENCH=92615
IDSOLVER=2326
FILE ID=node103/3743813-1338794734
RUNJOBID= node103-1338783950-5000
PBS_JOBID= 14636845
Free space on /tmp= 71056 MiB

SOLVER NAME= PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
BENCH NAME= PB12/normalized-PB12/OPT-BIGINT-LIN/sroussel/NG/normalized-NG.dot_unif_10null0mast_rapportOE2.0_TMAX60_K192_cod1.opb
COMMAND LINE= HOME/pbscip.linux.x86_64.gnu.opt.spx -f BENCHNAME -t TIMEOUT -m MEMLIMIT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3743813-1338794734/watcher-3743813-1338794734 -o /tmp/evaluation-result-3743813-1338794734/solver-3743813-1338794734 -C 1800 -W 1900 -M 15500  HOME/pbscip.linux.x86_64.gnu.opt.spx -f HOME/instance-3743813-1338794734.opb -t 1800 -m 15500

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 9306639b4fe31ed54dc3203c6f1fc129
RANDOM SEED=1492428745

node103.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.37
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5411.36
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.189
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        31143456 kB
Buffers:          206520 kB
Cached:           479588 kB
SwapCached:         5648 kB
Active:           515512 kB
Inactive:         439768 kB
Active(anon):     267176 kB
Inactive(anon):     4740 kB
Active(file):     248336 kB
Inactive(file):   435028 kB
Unevictable:        7116 kB
Mlocked:            7116 kB
SwapTotal:      67108856 kB
SwapFree:       67096352 kB
Dirty:             83204 kB
Writeback:             0 kB
AnonPages:        274420 kB
Mapped:            10424 kB
Shmem:                 0 kB
Slab:             636228 kB
SReclaimable:      76384 kB
SUnreclaim:       559844 kB
KernelStack:        1448 kB
PageTables:         4652 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     391016 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:      2048 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71044 MiB
End job on node103 at 2012-06-04 09:55:37