Trace number 3322503

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-industrial? (TO) 40000.8 5017.26

General information on the benchmark

Nameapplication/kullmann/AES/Benchmarks/
64/aes_64_4_keyfind_1.cnf
MD5SUM07ea67092ce3fd4529e72f69434e3f99
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1808
Number of clauses9968
Sum of the clauses size39152
Maximum clause length5
Minimum clause length1
Number of clauses of size 1144
Number of clauses of size 2320
Number of clauses of size 33904
Number of clauses of size 41344
Number of clauses of size 54256
Number of clauses of size over 50

Solver Data 

0.37/0.78	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.37/0.78	c INFO: solving HOME/instance-3322503-1304351098.cnf
0.37/0.78	c INFO: computing task features
0.43/0.82	c INFO: computed variable-clause graph statistics
0.43/0.82	c INFO: computed clause balance statistics
0.64/1.04	c INFO: computed variable balance statistics
0.74/1.18	c INFO: computed small-clause counts
0.74/1.18	c INFO: computed Horn-clause counts
0.74/1.19	c INFO: computed variable graph statistics
0.74/1.19	c INFO: collected features for HOME/instance-3322503-1304351098.cnf in 0.41 s
0.83/1.22	c INFO: running lingeling-276@0 for 97 with 39999 remaining (b = 0.28)
0.83/1.25	c INFO: running cryptominisat-2.9.0@0 for 485 with 39999 remaining (b = 0.35)
0.83/1.28	c INFO: running lingeling-276@0 for 1261 with 39999 remaining (b = 0.52)
0.93/1.31	c INFO: running lingeling-276@0 for 1261 with 39999 remaining (b = 0.46)
0.93/1.34	c INFO: running cryptominisat-2.9.0@0 for 485 with 39999 remaining (b = 0.26)
0.93/1.37	c INFO: running SApperloT@0 for 97 with 39999 remaining (b = 0.03)
1.03/1.40	c INFO: running precosat-570@0 for 388 with 39999 remaining (b = 0.20)
1.03/1.43	c INFO: running cryptominisat-2.9.0@0 for 1940 with 39999 remaining (b = 0.48)
800.50/101.43	c INFO: running lingeling-276@0 for 97 with 39899 remaining (b = 0.07)
802.10/101.60	c INFO: running lingeling-276@0 for 1261 with 39899 remaining (b = 0.34)
1560.43/196.63	c INFO: running cryptominisat-2.9.0@0 for 1940 with 39804 remaining (b = 0.42)
3104.99/390.14	c INFO: running precosat-570@0 for 1649 with 39610 remaining (b = 0.36)
3901.57/489.94	c INFO: running precosat-570@0 for 388 with 39510 remaining (b = 0.16)
3901.57/489.98	c INFO: running lingeling-276@100 for 485 with 39510 remaining (b = 0.17)
7052.76/884.71	c INFO: running precosat-570@487 for 291 with 39116 remaining (b = 0.13)
7795.92/977.85	c INFO: running precosat-570@0 for 194 with 39022 remaining (b = 0.11)
9347.51/1172.27	c INFO: running precosat-570@975 for 194 with 38828 remaining (b = 0.08)
9363.44/1174.27	c INFO: running precosat-570@286 for 97 with 38826 remaining (b = 0.04)
10108.92/1267.67	c INFO: running lingeling-276@0 for 97 with 38733 remaining (b = 0.04)
10171.17/1275.50	c INFO: running lingeling-276@494 for 97 with 38725 remaining (b = 0.04)
10299.68/1291.58	c INFO: running lingeling-276@0 for 97 with 38709 remaining (b = 0.03)
10902.28/1367.01	c INFO: running precosat-570@1553 for 97 with 38633 remaining (b = 0.03)
10908.65/1367.85	c INFO: running lingeling-276@0 for 1164 with 38632 remaining (b = 0.26)
10923.01/1369.68	c INFO: running lingeling-276@100 for 1164 with 38631 remaining (b = 0.26)
10947.82/1372.73	c INFO: running lingeling-276@95 for 97 with 38628 remaining (b = 0.03)
11076.27/1388.82	c INFO: running lingeling-276@99 for 1164 with 38611 remaining (b = 0.26)
11670.90/1463.34	c INFO: running precosat-570@2817 for 291 with 38537 remaining (b = 0.08)
11723.59/1469.97	c INFO: running precosat-570@1170 for 97 with 38530 remaining (b = 0.03)
12500.17/1567.20	c INFO: running cryptominisat-2.9.0@0 for 1843 with 38433 remaining (b = 0.34)
13958.35/1749.99	c INFO: running precosat-570@2914 for 97 with 38250 remaining (b = 0.04)
14766.88/1851.22	c INFO: running precosat-570@3015 for 97 with 38149 remaining (b = 0.03)
15526.65/1946.45	c INFO: running lingeling-276@192 for 1164 with 38054 remaining (b = 0.23)
15578.50/1952.93	c INFO: running precosat-570@0 for 97 with 38047 remaining (b = 0.06)
16331.12/2047.22	c INFO: running glucose@0 for 873 with 37953 remaining (b = 0.20)
16378.18/2053.18	c INFO: running precosat-570@0 for 291 with 37947 remaining (b = 0.09)
17121.99/2146.46	c INFO: running cryptominisat-2.9.0@388 for 97 with 37854 remaining (b = 0.04)
17913.57/2245.74	c INFO: running precosat-570@3110 for 97 with 37755 remaining (b = 0.01)
18688.53/2342.98	c INFO: running cryptominisat-2.9.0@0 for 97 with 37657 remaining (b = 0.02)
18743.44/2349.89	c INFO: running cryptominisat-2.9.0@487 for 1455 with 37650 remaining (b = 0.29)
19458.53/2439.58	c INFO: running cryptominisat-2.9.0@296 for 97 with 37561 remaining (b = 0.04)
20250.19/2538.88	c INFO: running cryptominisat-2.9.0@100 for 388 with 37461 remaining (b = 0.11)
20262.86/2540.43	c INFO: running cryptominisat-2.9.0@1461 for 194 with 37460 remaining (b = 0.06)
20267.68/2541.06	c INFO: running precosat-570@1944 for 291 with 37459 remaining (b = 0.07)
20355.37/2552.09	c INFO: running SApperloT@0 for 291 with 37448 remaining (b = 0.07)
21779.06/2730.65	c INFO: running cryptominisat-2.9.0@0 for 194 with 37270 remaining (b = 0.06)
22568.26/2829.68	c INFO: running cryptominisat-2.9.0@2626 for 97 with 37171 remaining (b = 0.02)
22663.91/2841.69	c INFO: running precosat-570@0 for 291 with 37159 remaining (b = 0.06)
23336.73/2926.02	c INFO: running cryptominisat-2.9.0@0 for 1940 with 37074 remaining (b = 0.28)
23360.56/2929.01	c INFO: running glucose@99 for 97 with 37071 remaining (b = 0.02)
23363.73/2929.45	c INFO: running glucose@0 for 873 with 37071 remaining (b = 0.14)
23384.48/2932.09	c INFO: running cryptominisat-2.9.0@2915 for 97 with 37068 remaining (b = 0.02)
24136.19/3026.36	c INFO: running glucose@490 for 388 with 36974 remaining (b = 0.08)
24144.14/3027.36	c INFO: running cryptominisat-2.9.0@3012 for 291 with 36973 remaining (b = 0.05)
24824.93/3112.74	c INFO: running cryptominisat-2.9.0@287 for 291 with 36888 remaining (b = 0.05)
24997.09/3134.35	c INFO: running lingeling-276@196 for 388 with 36866 remaining (b = 0.07)
26476.14/3320.09	c INFO: running cryptominisat-2.9.0@4179 for 291 with 36680 remaining (b = 0.05)
27167.27/3406.78	c INFO: running lingeling-276@196 for 1067 with 36594 remaining (b = 0.16)
27242.13/3416.19	c INFO: running lingeling-276@195 for 1067 with 36584 remaining (b = 0.16)
27380.84/3433.58	c INFO: running glucosER@0 for 194 with 36567 remaining (b = 0.03)
28118.91/3526.19	c INFO: running lingeling-276@488 for 776 with 36474 remaining (b = 0.13)
28806.85/3612.44	c INFO: running glucose@584 for 582 with 36388 remaining (b = 0.09)
28947.13/3630.05	c INFO: running glucosER@390 for 582 with 36370 remaining (b = 0.11)
30391.50/3811.24	c INFO: running precosat-570@487 for 291 with 36189 remaining (b = 0.05)
30550.12/3831.21	c INFO: running precosat-570@874 for 388 with 36169 remaining (b = 0.06)
32748.54/4106.91	c INFO: running glucose@101 for 97 with 35893 remaining (b = 0.02)
33463.58/4196.68	c INFO: running glucose@780 for 97 with 35804 remaining (b = 0.04)
33523.39/4204.19	c INFO: running glucose@684 for 485 with 35796 remaining (b = 0.10)
33627.74/4217.30	c INFO: running glucosER@1269 for 582 with 35783 remaining (b = 0.13)
33673.96/4223.06	c INFO: running lingeling-276@0 for 194 with 35777 remaining (b = 0.03)
34270.94/4297.96	c INFO: running glucose@877 for 291 with 35702 remaining (b = 0.05)
34388.87/4312.76	c INFO: running lingeling-276@779 for 97 with 35688 remaining (b = 0.02)
35148.48/4408.02	c INFO: running lingeling-276@874 for 388 with 35592 remaining (b = 0.07)
35240.10/4419.54	c INFO: running precosat-570@1169 for 97 with 35581 remaining (b = 0.02)
35690.36/4476.05	c INFO: running precosat-570@1938 for 388 with 35524 remaining (b = 0.06)
35868.78/4498.45	c INFO: running lingeling-276@585 for 679 with 35502 remaining (b = 0.10)
36015.42/4516.82	c INFO: running lingeling-276@2037 for 97 with 35484 remaining (b = 0.02)
36563.60/4585.61	c INFO: running glucose@1652 for 194 with 35415 remaining (b = 0.03)
36790.75/4614.10	c INFO: running lingeling-276@100 for 485 with 35386 remaining (b = 0.07)
37484.98/4701.27	c INFO: running glucose@1269 for 97 with 35299 remaining (b = 0.02)
38136.90/4783.08	c INFO: running precosat-570@0 for 291 with 35217 remaining (b = 0.05)
38244.41/4796.55	c INFO: running glucose@1465 for 97 with 35204 remaining (b = 0.02)
38292.23/4802.58	c INFO: running glucosER@1660 for 194 with 35198 remaining (b = 0.09)
38302.49/4803.88	c INFO: running lingeling-276@1069 for 97 with 35196 remaining (b = 0.02)
38749.18/4859.94	c INFO: running precosat-570@3007 for 97 with 35140 remaining (b = 0.02)
38852.72/4872.97	c INFO: running SApperloT@189 for 485 with 35127 remaining (b = 0.07)
39054.17/4898.24	c INFO: running glucose@1749 for 97 with 35102 remaining (b = 0.01)
39082.77/4901.87	c INFO: running glucose@2049 for 97 with 35098 remaining (b = 0.02)
39548.50/4960.31	c INFO: running precosat-570@4950 for 97 with 35040 remaining (b = 0.03)
39808.80/4993.07	c INFO: running cryptominisat-2.9.0@0 for 97 with 35007 remaining (b = 0.01)
39828.72/4995.53	c INFO: running cryptominisat-2.9.0@583 for 1358 with 35005 remaining (b = 0.17)
39833.55/4996.15	c INFO: running glucose@2149 for 291 with 35004 remaining (b = 0.04)
40000.84/5017.23	c INFO: running precosat-570@3106 for 97 with 34983 remaining (b = 0.01)
40000.84/5017.23	Traceback (most recent call last):
40000.84/5017.23	  File "HOME/solve", line 33, in <module>
40000.84/5017.23	    plac.call(main)
40000.84/5017.23	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call
40000.84/5017.23	    cmd, result = parser_from(obj).consume(arglist)
40000.84/5017.23	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 192, in consume
40000.84/5017.23	    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
40000.84/5017.23	  File "HOME/borg/build/lib/borg/tools/solve.py", line 84, in main
40000.84/5017.23	    (_, answer, _) = solver(input_path, budget, cores)
40000.84/5017.23	  File "HOME/borg/build/lib/borg/bilevel.py", line 202, in __call__
40000.84/5017.23	    solver.go(max_cost * borg.defaults.machine_speed)
40000.84/5017.23	  File "HOME/borg/build/lib/borg/solvers.py", line 179, in go
40000.84/5017.23	    self._process.start()
40000.84/5017.23	  File "/home/evaluation/evaluation/tools64/lib/python2.7/multiprocessing/process.py", line 94, in start
40000.84/5017.24	    assert self._popen is None, 'cannot start a process twice'
40000.84/5017.24	AssertionError: cannot start a process twice

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data

Sorry, but this trace is too large and cannot be obtained.

Launcher Data 

Begin job on node117 at 2011-05-02 17:44:58
IDJOB=3322503
IDBENCH=83416
IDSOLVER=1710
FILE ID=node117/3322503-1304351098
RUNJOBID= node117-1304349834-20706
PBS_JOBID= 13178674
Free space on /tmp= 72680 MiB

SOLVER NAME= borg-sat 11.03.02-industrial
BENCH NAME= SAT11/application/kullmann/AES/Benchmarks/64/aes_64_4_keyfind_1.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322503-1304351098/watcher-3322503-1304351098 -o /tmp/evaluation-result-3322503-1304351098/solver-3322503-1304351098 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3322503-1304351098.cnf -seed 1550510052 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 07ea67092ce3fd4529e72f69434e3f99
RANDOM SEED=1550510052

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      25278876 kB
Buffers:       1018052 kB
Cached:        4382340 kB
SwapCached:        160 kB
Active:        2209984 kB
Inactive:      4973524 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      25278876 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:           31732 kB
Writeback:           0 kB
AnonPages:     1783408 kB
Mapped:          20828 kB
Slab:           408472 kB
PageTables:      15484 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS: 35741632 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72648 MiB
End job on node117 at 2011-05-02 19:08:37