Trace number 3498542

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-industrial (fixed)UNSAT 4707.97 4708.03

General information on the benchmark

NameSAT-Race-2010-CNF/software-verification/
nec/hard-18-U-10652.cnf
MD5SUMf0f11935e861485056485b47c5e1a7c3
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark4.38633
Satisfiable
(Un)Satisfiability was proved
Number of variables1987351
Number of clauses5963534
Sum of the clauses size14253662
Maximum clause length1155
Minimum clause length1
Number of clauses of size 1109
Number of clauses of size 23971661
Number of clauses of size 31984242
Number of clauses of size 4766
Number of clauses of size 5765
Number of clauses of size over 55991

Solver Data 

0.36/0.68	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.36/0.68	c INFO: solving HOME/instance-3498542-1307958135.cnf
0.36/0.68	c INFO: computing task features
40.14/40.40	c INFO: computed variable-clause graph statistics
40.84/41.10	c INFO: computed clause balance statistics
129.55/129.85	c INFO: computed variable balance statistics
194.65/194.99	c INFO: computed small-clause counts
206.75/207.01	c INFO: computed Horn-clause counts
4666.99/4667.11	c INFO: computed variable graph statistics
4666.99/4667.13	c INFO: collected features for HOME/instance-3498542-1307958135.cnf in 4663.57 s
4666.99/4667.14	c INFO: running lingeling-276@0 for 97 with 336 remaining (b = 0.99)
4707.70/4707.81	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /var/tmp/evaluation-result-3498542-1307958135/watcher-3498542-1307958135 -o /var/tmp/evaluation-result-3498542-1307958135/solver-3498542-1307958135 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3498542-1307958135.cnf -seed 1246401360 -budget 5000 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.01 1.94 3/200 31122
/proc/meminfo: memFree=10492772/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=15184 CPUtime=0 cores=0,2,4,6
/proc/31122/stat : 31122 (python) S 31120 31122 30781 0 -1 4202496 725 0 0 0 0 0 0 0 22 0 1 0 986044577 15548416 615 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020393984 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/31122/statm: 3796 615 321 364 0 283 0

[startup+0.103488 s]
/proc/loadavg: 2.00 2.01 1.94 3/200 31122
/proc/meminfo: memFree=10492772/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=85620 CPUtime=0.06 cores=0,2,4,6
/proc/31122/stat : 31122 (python) S 31120 31122 30781 0 -1 4202496 2630 0 0 0 4 2 0 0 18 0 1 0 986044577 87674880 2348 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020393984 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/31122/statm: 21405 2348 713 364 0 1622 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85620

[startup+0.200399 s]
/proc/loadavg: 2.00 2.01 1.94 3/200 31122
/proc/meminfo: memFree=10492772/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=124068 CPUtime=0.12 cores=0,2,4,6
/proc/31122/stat : 31122 (python) S 31120 31122 30781 0 -1 4202496 4183 0 1 0 8 4 0 0 17 0 1 0 986044577 127045632 3795 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020393984 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/31122/statm: 31017 3795 991 364 0 2828 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 124068

[startup+0.300363 s]
/proc/loadavg: 2.00 2.01 1.94 3/200 31122
/proc/meminfo: memFree=10492772/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=135748 CPUtime=0.19 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 5969 932 1 0 14 5 0 0 17 0 1 0 986044577 139005952 4149 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020391941 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 33937 4149 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135748

[startup+0.700278 s]
/proc/loadavg: 2.00 2.01 1.94 3/200 31122
/proc/meminfo: memFree=10492772/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=399296 CPUtime=0.44 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 14680 932 1 0 30 14 0 0 16 0 1 0 986044577 408879104 12352 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020395776 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 99824 12372 1758 364 0 35748 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 399296

[startup+1.50115 s]
/proc/loadavg: 2.00 2.01 1.94 3/201 31127
/proc/meminfo: memFree=10328248/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=449184 CPUtime=1.24 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 51684 932 1 0 104 20 0 0 17 0 1 0 986044577 459964416 49227 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020056773 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 112296 49227 1761 364 0 48220 0
Current children cumulated CPU time (s) 1.24
Current children cumulated vsize (KiB) 449184

[startup+3.10082 s]
/proc/loadavg: 2.00 2.01 1.94 3/201 31127
/proc/meminfo: memFree=10252360/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=558572 CPUtime=2.84 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 78852 932 1 0 260 24 0 0 19 0 1 0 986044577 571977728 76395 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020056773 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 139643 76395 1761 364 0 75567 0
Current children cumulated CPU time (s) 2.84
Current children cumulated vsize (KiB) 558572

[startup+6.30026 s]
/proc/loadavg: 2.00 2.01 1.94 3/201 31127
/proc/meminfo: memFree=9986148/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=776288 CPUtime=6.04 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 133079 932 1 0 572 32 0 0 25 0 1 0 986044577 794918912 130622 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020056773 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 194072 130623 1761 364 0 129996 0
Current children cumulated CPU time (s) 6.04
Current children cumulated vsize (KiB) 776288

[startup+12.7011 s]
/proc/loadavg: 2.00 2.01 1.94 3/205 31132
/proc/meminfo: memFree=9583012/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1161864 CPUtime=12.45 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 228222 932 1 0 1201 44 0 0 25 0 1 0 986044577 1189748736 225765 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977243835 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 290466 225765 1761 364 0 226390 0
Current children cumulated CPU time (s) 12.45
Current children cumulated vsize (KiB) 1161864

[startup+25.5007 s]
/proc/loadavg: 2.00 2.01 1.94 3/205 31132
/proc/meminfo: memFree=9332264/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1437160 CPUtime=25.24 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 1015629 932 1 0 2400 124 0 0 25 0 1 0 986044577 1471651840 286325 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977417338 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 359290 286327 1814 364 0 295214 0
Current children cumulated CPU time (s) 25.24
Current children cumulated vsize (KiB) 1437160

[startup+51.1011 s]
/proc/loadavg: 2.06 2.02 1.95 3/205 31133
/proc/meminfo: memFree=9484424/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1251300 CPUtime=50.84 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2003514 932 1 0 4857 227 0 0 25 0 1 0 986044577 1281331200 250210 33554432000 4194304 5683644 140734992672432 18446744073709551615 4458802 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 312825 250210 1826 364 0 248749 0
Current children cumulated CPU time (s) 50.84
Current children cumulated vsize (KiB) 1251300

[startup+102.301 s]
/proc/loadavg: 2.02 2.02 1.95 3/205 31134
/proc/meminfo: memFree=9481584/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1251300 CPUtime=102.04 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2003514 932 1 0 9977 227 0 0 25 0 1 0 986044577 1281331200 250210 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797979878724 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 312825 250210 1826 364 0 248749 0
Current children cumulated CPU time (s) 102.04
Current children cumulated vsize (KiB) 1251300

[startup+162.301 s]
/proc/loadavg: 2.01 2.01 1.95 3/205 31136
/proc/meminfo: memFree=9480116/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1252324 CPUtime=162.05 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2011502 932 1 0 15977 228 0 0 25 0 1 0 986044577 1282379776 250467 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020087390 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313081 250467 1827 364 0 249005 0
Current children cumulated CPU time (s) 162.05
Current children cumulated vsize (KiB) 1252324

[startup+222.301 s]
/proc/loadavg: 2.00 2.00 1.95 3/206 31138
/proc/meminfo: memFree=9476848/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=222.04 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 21893 311 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004160 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 222.04
Current children cumulated vsize (KiB) 1253240

[startup+282.307 s]
/proc/loadavg: 2.00 2.00 1.95 3/206 31140
/proc/meminfo: memFree=9475820/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=282.05 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 27894 311 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004160 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 282.05
Current children cumulated vsize (KiB) 1253240

[startup+342.301 s]
/proc/loadavg: 2.00 2.00 1.95 3/206 31141
/proc/meminfo: memFree=9476120/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=342.05 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 33894 311 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004172 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 342.05
Current children cumulated vsize (KiB) 1253240

[startup+402.301 s]
/proc/loadavg: 2.00 2.00 1.95 3/206 31143
/proc/meminfo: memFree=9474056/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=402.05 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 39894 311 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004160 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 402.05
Current children cumulated vsize (KiB) 1253240

[startup+462.301 s]
/proc/loadavg: 2.00 2.00 1.95 3/206 31145
/proc/meminfo: memFree=9475376/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=462.05 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 45894 311 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004160 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 462.05
Current children cumulated vsize (KiB) 1253240

[startup+522.301 s]
/proc/loadavg: 2.00 2.00 1.95 3/206 31146
/proc/meminfo: memFree=9422852/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=522.05 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 51894 311 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004160 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 522.05
Current children cumulated vsize (KiB) 1253240

[startup+582.301 s]
/proc/loadavg: 2.00 2.00 1.95 3/206 31148
/proc/meminfo: memFree=9432080/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=582.06 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 57895 311 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004160 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 582.06
Current children cumulated vsize (KiB) 1253240

[startup+642.301 s]

################
# More data... #
################

/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 4302.18
Current children cumulated vsize (KiB) 1253240

[startup+4362.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/172 31290
/proc/meminfo: memFree=9937828/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=4362.18 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 435906 312 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004160 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 4362.18
Current children cumulated vsize (KiB) 1253240

[startup+4422.3 s]
/proc/loadavg: 1.06 1.02 1.00 2/172 31291
/proc/meminfo: memFree=9937896/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=4422.18 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 441906 312 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004172 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 4422.18
Current children cumulated vsize (KiB) 1253240

[startup+4482.3 s]
/proc/loadavg: 1.02 1.01 1.00 2/172 31293
/proc/meminfo: memFree=9937960/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=4482.18 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 447906 312 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004172 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 4482.18
Current children cumulated vsize (KiB) 1253240

[startup+4542.3 s]
/proc/loadavg: 1.01 1.00 1.00 2/172 31294
/proc/meminfo: memFree=9938020/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=4542.18 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 453906 312 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004172 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 4542.18
Current children cumulated vsize (KiB) 1253240

[startup+4602.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/172 31296
/proc/meminfo: memFree=9937828/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=4602.18 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 459906 312 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004172 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 4602.18
Current children cumulated vsize (KiB) 1253240

[startup+4662.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/172 31298
/proc/meminfo: memFree=9937760/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=1253240 CPUtime=4662.19 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2799884 932 1 0 465907 312 0 0 25 0 1 0 986044577 1283317760 250702 33554432000 4194304 5683644 140734992672432 18446744073709551615 47797977004160 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31122/statm: 313310 250702 1833 364 0 249234 0
Current children cumulated CPU time (s) 4662.19
Current children cumulated vsize (KiB) 1253240

Solver just ended. Dumping a history of the last processes samples

[startup+4685.4 s]
/proc/loadavg: 1.00 1.00 1.00 2/175 31301
/proc/meminfo: memFree=10058844/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=819064 CPUtime=4667.03 cores=0,2,4,6
/proc/31122/stat : 31122 (python) S 31120 31122 30781 0 -1 4202496 2811822 932 1 0 466387 316 0 0 25 0 2 0 986044577 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31122/statm: 204766 139661 1867 364 0 140684 0
[pid=31299] ppid=31122 vsize=808820 CPUtime=0.02 cores=0,2,4,6
/proc/31299/stat : 31299 (python) S 31122 31122 30781 0 -1 4202560 882 0 0 0 0 2 0 0 18 0 1 0 986511291 828231680 138166 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020416015 0 0 16781312 514 15161157039310209284 0 0 17 6 0 0 0
/proc/31299/statm: 202205 138166 376 364 0 138123 0
[pid=31301] ppid=31299 vsize=323344 CPUtime=18.21 cores=0,2,4,6
/proc/31301/stat : 31301 (lingeling) R 31299 31301 31301 0 -1 4202496 201537 0 0 0 1795 26 0 0 25 0 1 0 986511293 331104256 80004 33554432000 4194304 4352236 140735212578800 18446744073709551615 4290984 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/31301/statm: 80836 80004 127 39 0 79916 0
[pid=31122/tid=31300] ppid=31120 vsize=819064 CPUtime=0 cores=0,2,4,6
/proc/31122/task/31300/stat : 31300 (python) S 31120 31122 30781 0 -1 4202560 31 932 0 0 0 0 0 0 25 0 2 0 986511293 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032219569 0 0 16781312 2 18446604443920106752 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4685.26
Current children cumulated vsize (KiB) 1951228

[startup+4698.2 s]
/proc/loadavg: 1.00 1.00 1.00 2/175 31302
/proc/meminfo: memFree=9886464/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=819064 CPUtime=4667.03 cores=0,2,4,6
/proc/31122/stat : 31122 (python) S 31120 31122 30781 0 -1 4202496 2811822 932 1 0 466387 316 0 0 25 0 2 0 986044577 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31122/statm: 204766 139661 1867 364 0 140684 0
[pid=31299] ppid=31122 vsize=808820 CPUtime=0.04 cores=0,2,4,6
/proc/31299/stat : 31299 (python) S 31122 31122 30781 0 -1 4202560 887 0 0 0 1 3 0 0 18 0 1 0 986511291 828231680 138166 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020416015 0 0 16781312 514 15161157039310209284 0 0 17 6 0 0 0
/proc/31299/statm: 202205 138166 376 364 0 138123 0
[pid=31301] ppid=31299 vsize=495620 CPUtime=31.02 cores=0,2,4,6
/proc/31301/stat : 31301 (lingeling) R 31299 31301 31301 0 -1 4202496 244606 0 0 0 3072 30 0 0 25 0 1 0 986511293 507514880 123073 33554432000 4194304 4352236 140735212578800 18446744073709551615 4233386 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/31301/statm: 123905 123073 127 39 0 122985 0
[pid=31122/tid=31300] ppid=31120 vsize=819064 CPUtime=0 cores=0,2,4,6
/proc/31122/task/31300/stat : 31300 (python) S 31120 31122 30781 0 -1 4202560 31 932 0 0 0 0 0 0 25 0 2 0 986511293 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032219569 0 0 16781312 2 18446604443920106752 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4698.09
Current children cumulated vsize (KiB) 2123504

[startup+4704.6 s]
/proc/loadavg: 1.00 1.00 1.00 2/175 31302
/proc/meminfo: memFree=9949824/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=819064 CPUtime=4667.03 cores=0,2,4,6
/proc/31122/stat : 31122 (python) S 31120 31122 30781 0 -1 4202496 2811822 932 1 0 466387 316 0 0 25 0 2 0 986044577 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31122/statm: 204766 139661 1867 364 0 140684 0
[pid=31299] ppid=31122 vsize=808820 CPUtime=0.05 cores=0,2,4,6
/proc/31299/stat : 31299 (python) S 31122 31122 30781 0 -1 4202560 890 0 0 0 2 3 0 0 18 0 1 0 986511291 828231680 138166 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020416015 0 0 16781312 514 15161157039310209284 0 0 17 6 0 0 0
/proc/31299/statm: 202205 138166 376 364 0 138123 0
[pid=31301] ppid=31299 vsize=432436 CPUtime=37.42 cores=0,2,4,6
/proc/31301/stat : 31301 (lingeling) R 31299 31301 31301 0 -1 4202496 263505 0 0 0 3710 32 0 0 25 0 1 0 986511293 442814464 107279 33554432000 4194304 4352236 140735212578800 18446744073709551615 4327902 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/31301/statm: 108109 107279 129 39 0 107189 0
[pid=31122/tid=31300] ppid=31120 vsize=819064 CPUtime=0 cores=0,2,4,6
/proc/31122/task/31300/stat : 31300 (python) S 31120 31122 30781 0 -1 4202560 31 932 0 0 0 0 0 0 25 0 2 0 986511293 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032219569 0 0 16781312 2 18446604443920106752 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4704.5
Current children cumulated vsize (KiB) 2060320

[startup+4706.2 s]
/proc/loadavg: 1.00 1.00 1.00 2/175 31302
/proc/meminfo: memFree=9949824/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=819064 CPUtime=4667.03 cores=0,2,4,6
/proc/31122/stat : 31122 (python) S 31120 31122 30781 0 -1 4202496 2811822 932 1 0 466387 316 0 0 25 0 2 0 986044577 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31122/statm: 204766 139661 1867 364 0 140684 0
[pid=31299] ppid=31122 vsize=808820 CPUtime=0.05 cores=0,2,4,6
/proc/31299/stat : 31299 (python) S 31122 31122 30781 0 -1 4202560 890 0 0 0 2 3 0 0 18 0 1 0 986511291 828231680 138166 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020416015 0 0 16781312 514 15161157039310209284 0 0 17 6 0 0 0
/proc/31299/statm: 202205 138166 376 364 0 138123 0
[pid=31301] ppid=31299 vsize=432436 CPUtime=39.02 cores=0,2,4,6
/proc/31301/stat : 31301 (lingeling) R 31299 31301 31301 0 -1 4202496 263505 0 0 0 3870 32 0 0 25 0 1 0 986511293 442814464 107279 33554432000 4194304 4352236 140735212578800 18446744073709551615 4208062 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/31301/statm: 108109 107279 129 39 0 107189 0
[pid=31122/tid=31300] ppid=31120 vsize=819064 CPUtime=0 cores=0,2,4,6
/proc/31122/task/31300/stat : 31300 (python) S 31120 31122 30781 0 -1 4202560 31 932 0 0 0 0 0 0 25 0 2 0 986511293 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032219569 0 0 16781312 2 18446604443920106752 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4706.1
Current children cumulated vsize (KiB) 2060320

[startup+4707 s]
/proc/loadavg: 1.00 1.00 1.00 2/175 31302
/proc/meminfo: memFree=9949824/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=819064 CPUtime=4667.03 cores=0,2,4,6
/proc/31122/stat : 31122 (python) S 31120 31122 30781 0 -1 4202496 2811822 932 1 0 466387 316 0 0 25 0 2 0 986044577 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31122/statm: 204766 139661 1867 364 0 140684 0
[pid=31299] ppid=31122 vsize=808820 CPUtime=0.05 cores=0,2,4,6
/proc/31299/stat : 31299 (python) S 31122 31122 30781 0 -1 4202560 890 0 0 0 2 3 0 0 18 0 1 0 986511291 828231680 138166 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020416015 0 0 16781312 514 15161157039310209284 0 0 17 6 0 0 0
/proc/31299/statm: 202205 138166 376 364 0 138123 0
[pid=31301] ppid=31299 vsize=432436 CPUtime=39.82 cores=0,2,4,6
/proc/31301/stat : 31301 (lingeling) R 31299 31301 31301 0 -1 4202496 263505 0 0 0 3950 32 0 0 25 0 1 0 986511293 442814464 107279 33554432000 4194304 4352236 140735212578800 18446744073709551615 4327902 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/31301/statm: 108109 107279 129 39 0 107189 0
[pid=31122/tid=31300] ppid=31120 vsize=819064 CPUtime=0 cores=0,2,4,6
/proc/31122/task/31300/stat : 31300 (python) S 31120 31122 30781 0 -1 4202560 31 932 0 0 0 0 0 0 25 0 2 0 986511293 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032219569 0 0 16781312 2 18446604443920106752 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4706.9
Current children cumulated vsize (KiB) 2060320

[startup+4707.8 s]
/proc/loadavg: 1.00 1.00 1.00 2/175 31302
/proc/meminfo: memFree=9949824/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=819064 CPUtime=4667.03 cores=0,2,4,6
/proc/31122/stat : 31122 (python) S 31120 31122 30781 0 -1 4202496 2811822 932 1 0 466387 316 0 0 25 0 2 0 986044577 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/31122/statm: 204766 139661 1867 364 0 140684 0
[pid=31299] ppid=31122 vsize=808820 CPUtime=0.05 cores=0,2,4,6
/proc/31299/stat : 31299 (python) S 31122 31122 30781 0 -1 4202560 894 0 0 0 2 3 0 0 15 0 1 0 986511291 828231680 138166 33554432000 4194304 5683644 140734992672432 18446744073709551615 260020416015 0 0 16781312 514 3013440318633640196 0 0 17 6 0 0 0
/proc/31299/statm: 202205 138166 376 364 0 138123 0
[pid=31301] ppid=31299 vsize=432440 CPUtime=40.62 cores=0,2,4,6
/proc/31301/stat : 31301 (lingeling) R 31299 31301 31301 0 -1 4202496 263507 0 0 0 4029 33 0 0 25 0 1 0 986511293 442818560 43956 33554432000 4194304 4352236 140735212578800 18446744073709551615 260020439175 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31301/statm: 108110 43793 130 39 0 107190 0
[pid=31122/tid=31300] ppid=31120 vsize=819064 CPUtime=0 cores=0,2,4,6
/proc/31122/task/31300/stat : 31300 (python) S 31120 31122 30781 0 -1 4202560 31 932 0 0 0 0 0 0 25 0 2 0 986511293 838721536 139661 33554432000 4194304 5683644 140734992672432 18446744073709551615 260032219569 0 0 16781312 2 18446604443920106752 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4707.7
Current children cumulated vsize (KiB) 2060324

[startup+4708 s]
/proc/loadavg: 1.00 1.00 1.00 2/175 31302
/proc/meminfo: memFree=9949824/32951124 swapFree=67056304/67111528
[pid=31122] ppid=31120 vsize=819040 CPUtime=4707.92 cores=0,2,4,6
/proc/31122/stat : 31122 (python) R 31120 31122 30781 0 -1 4202496 2937298 265641 1 0 466398 324 4031 39 18 0 1 0 986044577 838696960 139660 33554432000 4194304 5683644 140734992672432 18446744073709551615 4570176 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31122/statm: 204760 139660 1866 364 0 140684 0
Current children cumulated CPU time (s) 4707.92
Current children cumulated vsize (KiB) 819040

Child status: 0
Real time (s): 4708.03
CPU time (s): 4707.97
CPU user time (s): 4704.31
CPU system time (s): 3.66644
CPU usage (%): 99.9989
Max. virtual memory (cumulated for all children) (KiB): 2156272

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4704.31
system time used= 3.66644
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3203114
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2926
involuntary context switches= 7117

runsolver used 7.62284 second user time and 14.7528 second system time

The end

Launcher Data 

Begin job on node110 at 2011-06-13 11:42:20
IDJOB=3498542
IDBENCH=88172
IDSOLVER=1991
FILE ID=node110/3498542-1307958135
RUNJOBID= node110-1307953127-30799
PBS_JOBID= 13559718
Free space on /var/tmp= 2732 MiB

SOLVER NAME= borg-sat 11.03.02-seq-industrial (fixed)
BENCH NAME= SAT-Race-2010-CNF/software-verification/nec/hard-18-U-10652.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /var/tmp/evaluation-result-3498542-1307958135/watcher-3498542-1307958135 -o /var/tmp/evaluation-result-3498542-1307958135/solver-3498542-1307958135 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3498542-1307958135.cnf -seed 1246401360 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f0f11935e861485056485b47c5e1a7c3
RANDOM SEED=1246401360

node110.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      10493704 kB
Buffers:         98124 kB
Cached:       21205664 kB
SwapCached:       9184 kB
Active:        6328224 kB
Inactive:     15398364 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      10493704 kB
SwapTotal:    67111528 kB
SwapFree:     67056304 kB
Dirty:          152828 kB
Writeback:           0 kB
AnonPages:      420016 kB
Mapped:          20028 kB
Slab:           661620 kB
PageTables:       9612 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  1050948 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /var/tmp at the end= 2732 MiB
End job on node110 at 2011-06-13 13:00:48