Trace number 3494708

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Haifa-MUC 2011-04-07UNSAT 2.44563 2.44681

General information on the benchmark

Namemus/marques-silva/design-debugging/
wb_conmax3.dimacs.filtered.cnf
MD5SUM70661d58e23d97579ff78155ba3338cb
Bench CategoryMUS (MUS)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark0.988849
Satisfiable
(Un)Satisfiability was proved
Number of variables277950
Number of clauses1221020
Sum of the clauses size3429264
Maximum clause length3
Minimum clause length1
Number of clauses of size 17116
Number of clauses of size 2219564
Number of clauses of size 3994340
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

2.39/2.43	c WARNING: for repeatability, setting FPU to use double precision
2.39/2.43	c iter 1 time 1.18082 not-muc 1220990 unknown 30 muc 0
2.39/2.43	c iter 2 time 1.72674 not-muc 1220990 unknown 29 muc 1
2.39/2.43	c iter 3 time 1.74573 not-muc 1220990 unknown 28 muc 2
2.39/2.43	c iter 4 time 1.76373 not-muc 1220990 unknown 27 muc 3
2.39/2.43	c iter 5 time 1.78173 not-muc 1220990 unknown 26 muc 4
2.39/2.43	c iter 6 time 1.80073 not-muc 1220990 unknown 25 muc 5
2.39/2.43	c iter 7 time 1.81972 not-muc 1220990 unknown 24 muc 6
2.39/2.43	c iter 8 time 1.83872 not-muc 1220990 unknown 23 muc 7
2.39/2.43	c iter 9 time 1.85772 not-muc 1220990 unknown 22 muc 8
2.39/2.43	c iter 10 time 1.87671 not-muc 1220990 unknown 21 muc 9
2.39/2.43	c iter 11 time 1.89571 not-muc 1220990 unknown 20 muc 10
2.39/2.43	c iter 12 time 1.91471 not-muc 1220990 unknown 19 muc 11
2.39/2.43	c iter 13 time 1.93271 not-muc 1220990 unknown 18 muc 12
2.39/2.43	c iter 14 time 1.9517 not-muc 1220990 unknown 17 muc 13
2.39/2.43	c iter 15 time 1.9707 not-muc 1220990 unknown 16 muc 14
2.39/2.43	c iter 16 time 1.9897 not-muc 1220990 unknown 15 muc 15
2.39/2.43	c iter 17 time 2.00869 not-muc 1220990 unknown 14 muc 16
2.39/2.43	c iter 18 time 2.02769 not-muc 1220990 unknown 13 muc 17
2.39/2.43	c iter 19 time 2.04669 not-muc 1220990 unknown 12 muc 18
2.39/2.43	c iter 20 time 2.06569 not-muc 1220990 unknown 11 muc 19
2.39/2.43	c iter 21 time 2.08468 not-muc 1220990 unknown 10 muc 20
2.39/2.43	c iter 22 time 2.10368 not-muc 1220990 unknown 9 muc 21
2.39/2.43	c iter 23 time 2.13567 not-muc 1220990 unknown 8 muc 22
2.39/2.43	c iter 24 time 2.15967 not-muc 1220990 unknown 7 muc 23
2.39/2.43	c iter 25 time 2.18367 not-muc 1220990 unknown 6 muc 24
2.39/2.43	c iter 26 time 2.21066 not-muc 1220990 unknown 5 muc 25
2.39/2.43	c iter 27 time 2.23466 not-muc 1220990 unknown 4 muc 26
2.39/2.43	c iter 28 time 2.26166 not-muc 1220990 unknown 3 muc 27
2.39/2.43	c iter 29 time 2.28565 not-muc 1220990 unknown 2 muc 28
2.39/2.43	c iter 30 time 2.31165 not-muc 1220990 unknown 1 muc 29
2.39/2.43	c final iter 31 time 2.33864 not-muc 1220990 unknown 0 muc 30
2.39/2.43	v 193 198 862 933 1004 1075 1146 1217 1283 1354 1081276 1081282 1081288 1081295 1081300 1081447 1165664 1167977 1167978 1169331 1182404 1186598 1190792 1194986 1199180 1203374 1207578 1211772 1216655 1216665 0
2.39/2.43	s UNSATISFIABLE

Verifier Data 

OK	30

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 988)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3494708-1307804742/watcher-3494708-1307804742 -o /tmp/evaluation-result-3494708-1307804742/solver-3494708-1307804742 -C 1800 -W 1850 -M 7680 HOME/hmuc HOME/instance-3494708-1307804742.cnf 

running on 2 cores: 1,3

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1850 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.97 3.28 2.83 5/189 16567
/proc/meminfo: memFree=29335076/32950928 swapFree=67051096/67111528
[pid=16567] ppid=16565 vsize=27212 CPUtime=0 cores=1,3
/proc/16567/stat : 16567 (hmuc) R 16565 16567 14391 0 -1 4202496 1126 0 0 0 0 0 0 0 24 0 1 0 970704977 27865088 1069 33554432000 4194304 4337479 140736974537648 18446744073709551615 4319585 0 0 4096 8388610 0 0 0 17 1 0 0 0
/proc/16567/statm: 6803 1072 215 35 0 3426 0

[startup+0.0892349 s]
/proc/loadavg: 3.97 3.28 2.83 5/189 16567
/proc/meminfo: memFree=29335076/32950928 swapFree=67051096/67111528
[pid=16567] ppid=16565 vsize=58256 CPUtime=0.08 cores=1,3
/proc/16567/stat : 16567 (hmuc) R 16565 16567 14391 0 -1 4202496 9475 0 0 0 7 1 0 0 24 0 1 0 970704977 59654144 9223 33554432000 4194304 4337479 140736974537648 18446744073709551615 251367546624 0 0 4096 8388610 0 0 0 17 1 0 0 0
/proc/16567/statm: 14564 9227 220 35 0 11187 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 58256

[startup+0.101223 s]
/proc/loadavg: 3.97 3.28 2.83 5/189 16567
/proc/meminfo: memFree=29335076/32950928 swapFree=67051096/67111528
[pid=16567] ppid=16565 vsize=59228 CPUtime=0.09 cores=1,3
/proc/16567/stat : 16567 (hmuc) R 16565 16567 14391 0 -1 4202496 10108 0 0 0 8 1 0 0 24 0 1 0 970704977 60649472 9765 33554432000 4194304 4337479 140736974537648 18446744073709551615 4321305 0 0 4096 8388610 0 0 0 17 1 0 0 0
/proc/16567/statm: 14807 9765 220 35 0 11430 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 59228

[startup+0.300181 s]
/proc/loadavg: 3.97 3.28 2.83 5/189 16567
/proc/meminfo: memFree=29335076/32950928 swapFree=67051096/67111528
[pid=16567] ppid=16565 vsize=93304 CPUtime=0.29 cores=1,3
/proc/16567/stat : 16567 (hmuc) R 16565 16567 14391 0 -1 4202496 18943 0 0 0 26 3 0 0 25 0 1 0 970704977 95543296 17924 33554432000 4194304 4337479 140736974537648 18446744073709551615 251367546624 0 0 4096 8388610 0 0 0 17 1 0 0 0
/proc/16567/statm: 23326 17928 220 35 0 19949 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 93304

[startup+0.701107 s]
/proc/loadavg: 3.97 3.28 2.83 5/189 16567
/proc/meminfo: memFree=29335076/32950928 swapFree=67051096/67111528
[pid=16567] ppid=16565 vsize=177648 CPUtime=0.69 cores=1,3
/proc/16567/stat : 16567 (hmuc) R 16565 16567 14391 0 -1 4202496 36223 0 0 0 63 6 0 0 25 0 1 0 970704977 181911552 33962 33554432000 4194304 4337479 140736974537648 18446744073709551615 4258524 0 0 4096 8388610 0 0 0 17 1 0 0 0
/proc/16567/statm: 44412 33964 220 35 0 41035 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 177648

[startup+1.50096 s]
/proc/loadavg: 3.97 3.28 2.83 5/190 16568
/proc/meminfo: memFree=29159944/32950928 swapFree=67051096/67111528
[pid=16567] ppid=16565 vsize=221460 CPUtime=1.49 cores=1,3
/proc/16567/stat : 16567 (hmuc) R 16565 16567 14391 0 -1 4202496 60693 0 0 0 140 9 0 0 25 0 1 0 970704977 226775040 46003 33554432000 4194304 4337479 140736974537648 18446744073709551615 4260835 0 0 4096 8388610 0 0 0 17 1 0 0 0
/proc/16567/statm: 55365 46003 256 35 0 51988 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 221460

Solver just ended. Dumping a history of the last processes samples

[startup+1.90089 s]
/proc/loadavg: 3.97 3.28 2.83 5/190 16568
/proc/meminfo: memFree=29159944/32950928 swapFree=67051096/67111528
[pid=16567] ppid=16565 vsize=232152 CPUtime=1.89 cores=1,3
/proc/16567/stat : 16567 (hmuc) R 16565 16567 14391 0 -1 4202496 63705 0 0 0 180 9 0 0 25 0 1 0 970704977 237723648 49015 33554432000 4194304 4337479 140736974537648 18446744073709551615 4309943 0 0 4096 8388610 0 0 0 17 1 0 0 0
/proc/16567/statm: 58038 49015 256 35 0 54661 0
Current children cumulated CPU time (s) 1.89
Current children cumulated vsize (KiB) 232152

[startup+2.30081 s]
/proc/loadavg: 3.98 3.30 2.84 5/190 16568
/proc/meminfo: memFree=29122620/32950928 swapFree=67051096/67111528
[pid=16567] ppid=16565 vsize=201800 CPUtime=2.29 cores=1,3
/proc/16567/stat : 16567 (hmuc) R 16565 16567 14391 0 -1 4202496 63705 0 0 0 220 9 0 0 25 0 1 0 970704977 206643200 44615 33554432000 4194304 4337479 140736974537648 18446744073709551615 4281720 0 0 4096 8388610 0 0 0 17 1 0 0 0
/proc/16567/statm: 50450 44615 256 35 0 47073 0
Current children cumulated CPU time (s) 2.29
Current children cumulated vsize (KiB) 201800

[startup+2.4008 s]
/proc/loadavg: 3.98 3.30 2.84 5/190 16568
/proc/meminfo: memFree=29122620/32950928 swapFree=67051096/67111528
[pid=16567] ppid=16565 vsize=201800 CPUtime=2.39 cores=1,3
/proc/16567/stat : 16567 (hmuc) R 16565 16567 14391 0 -1 4202496 63705 0 0 0 230 9 0 0 25 0 1 0 970704977 206643200 44615 33554432000 4194304 4337479 140736974537648 18446744073709551615 4281776 0 0 4096 8388610 0 0 0 17 1 0 0 0
/proc/16567/statm: 50450 44615 256 35 0 47073 0
Current children cumulated CPU time (s) 2.39
Current children cumulated vsize (KiB) 201800

Child status: 20
Real time (s): 2.44681
CPU time (s): 2.44563
CPU user time (s): 2.33864
CPU system time (s): 0.106983
CPU usage (%): 99.9516
Max. virtual memory (cumulated for all children) (KiB): 232152

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.33864
system time used= 0.106983
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 63706
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4
involuntary context switches= 2

runsolver used 0.005999 second user time and 0.012998 second system time

The end

Launcher Data 

Begin job on node113 at 2011-06-11 17:05:43
IDJOB=3494708
IDBENCH=89442
IDSOLVER=1985
FILE ID=node113/3494708-1307804742
RUNJOBID= node113-1307804398-15695
PBS_JOBID= 13551039
Free space on /tmp= 73932 MiB

SOLVER NAME= Haifa-MUC 2011-04-07
BENCH NAME= SAT11/mus/marques-silva/design-debugging/wb_conmax3.dimacs.filtered.cnf
COMMAND LINE= HOME/hmuc BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3494708-1307804742/watcher-3494708-1307804742 -o /tmp/evaluation-result-3494708-1307804742/solver-3494708-1307804742 -C 1800 -W 1850 -M 7680  HOME/hmuc HOME/instance-3494708-1307804742.cnf

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= 70661d58e23d97579ff78155ba3338cb
RANDOM SEED=428973533

node113.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      29335620 kB
Buffers:        277492 kB
Cached:        1175448 kB
SwapCached:      14384 kB
Active:        2624536 kB
Inactive:       780836 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      29335620 kB
SwapTotal:    67111528 kB
SwapFree:     67051096 kB
Dirty:          152648 kB
Writeback:          56 kB
AnonPages:     1949960 kB
Mapped:           9660 kB
Slab:           140540 kB
PageTables:       9308 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  2357132 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73932 MiB
End job on node113 at 2011-06-11 17:05:46