Trace number 3493730

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PicoMUS 941No Cert. 1800.07 1800.01

General information on the benchmark

Nameapplication/jarvisalo/HWMCC10-combination-timeframe-expansion/
hwmcc10-timeframe-expansion-k50-viscoherencep3-tseitin.cnf
MD5SUMe6bb96c7140e2a8d07499de24fcd1583
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables56416
Number of clauses168347
Sum of the clauses size392807
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2112230
Number of clauses of size 356115
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c [picomus] WARNING: no output file given
0.00/0.00	c [picomus] p cnf 56416 168347
57.00/57.06	s UNSATISFIABLE
57.10/57.17	c [picomus] extracted core 1 of size 104849 out of 168347 (62%) after 57.2 sec
106.19/106.25	c [picomus] extracted core 2 of size 88328 out of 168347 (52%) after 106.3 sec
146.79/146.83	c [picomus] extracted core 3 of size 80971 out of 168347 (48%) after 146.8 sec
174.80/174.86	c [picomus] extracted core 4 of size 76322 out of 168347 (45%) after 174.9 sec
199.90/199.98	c [picomus] extracted core 5 of size 73265 out of 168347 (44%) after 200.0 sec
242.70/242.76	c [picomus] extracted core 6 of size 71767 out of 168347 (43%) after 242.8 sec
266.00/266.09	c [picomus] extracted core 7 of size 70485 out of 168347 (42%) after 266.1 sec
292.90/292.96	c [picomus] extracted core 8 of size 69566 out of 168347 (41%) after 293.0 sec
324.50/324.53	c [picomus] extracted core 9 of size 69010 out of 168347 (41%) after 324.5 sec
355.91/355.97	c [picomus] extracted core 10 of size 68660 out of 168347 (41%) after 356.0 sec
394.00/394.03	c [picomus] extracted core 11 of size 68260 out of 168347 (41%) after 394.0 sec
423.52/423.57	c [picomus] extracted core 12 of size 67861 out of 168347 (40%) after 423.6 sec
454.11/454.14	c [picomus] extracted core 13 of size 67543 out of 168347 (40%) after 454.2 sec
476.61/476.63	c [picomus] extracted core 14 of size 67244 out of 168347 (40%) after 476.7 sec
509.01/509.03	c [picomus] extracted core 15 of size 66942 out of 168347 (40%) after 509.1 sec
540.31/540.40	c [picomus] extracted core 16 of size 66614 out of 168347 (40%) after 540.4 sec
561.81/561.83	c [picomus] extracted core 17 of size 66145 out of 168347 (39%) after 561.9 sec
603.61/603.61	c [picomus] extracted core 18 of size 66047 out of 168347 (39%) after 603.6 sec
630.21/630.21	c [picomus] extracted core 19 of size 65881 out of 168347 (39%) after 630.2 sec
653.11/653.18	c [picomus] extracted core 20 of size 65712 out of 168347 (39%) after 653.2 sec
683.61/683.62	c [picomus] extracted core 21 of size 65495 out of 168347 (39%) after 683.7 sec
716.72/716.71	c [picomus] extracted core 22 of size 65323 out of 168347 (39%) after 716.7 sec
755.01/755.08	c [picomus] extracted core 23 of size 65232 out of 168347 (39%) after 755.1 sec
778.51/778.57	c [picomus] extracted core 24 of size 65007 out of 168347 (39%) after 778.6 sec
813.52/813.57	c [picomus] extracted core 25 of size 64902 out of 168347 (39%) after 813.6 sec
852.82/852.83	c [picomus] extracted core 26 of size 64830 out of 168347 (39%) after 852.9 sec
888.83/888.85	c [picomus] extracted core 27 of size 64794 out of 168347 (38%) after 888.9 sec
914.22/914.22	c [picomus] extracted core 28 of size 64694 out of 168347 (38%) after 914.3 sec
962.73/962.79	c [picomus] extracted core 29 of size 64663 out of 168347 (38%) after 962.8 sec
985.53/985.56	c [picomus] extracted core 30 of size 64524 out of 168347 (38%) after 985.6 sec
1015.13/1015.12	c [picomus] extracted core 31 of size 64384 out of 168347 (38%) after 1015.2 sec
1039.43/1039.44	c [picomus] extracted core 32 of size 64284 out of 168347 (38%) after 1039.5 sec
1087.03/1087.01	c [picomus] extracted core 33 of size 64156 out of 168347 (38%) after 1087.1 sec
1110.83/1110.83	c [picomus] extracted core 34 of size 64057 out of 168347 (38%) after 1110.9 sec
1141.83/1141.81	c [picomus] extracted core 35 of size 63963 out of 168347 (38%) after 1141.9 sec
1195.34/1195.31	c [picomus] extracted core 36 of size 63948 out of 168347 (38%) after 1195.4 sec
1224.24/1224.29	c [picomus] extracted core 37 of size 63894 out of 168347 (38%) after 1224.3 sec
1253.53/1253.51	c [picomus] extracted core 38 of size 63775 out of 168347 (38%) after 1253.6 sec
1288.73/1288.72	c [picomus] extracted core 39 of size 63731 out of 168347 (38%) after 1288.8 sec
1318.53/1318.54	c [picomus] extracted core 40 of size 63646 out of 168347 (38%) after 1318.6 sec
1352.74/1352.72	c [picomus] extracted core 41 of size 63580 out of 168347 (38%) after 1352.8 sec
1385.64/1385.64	c [picomus] extracted core 42 of size 63556 out of 168347 (38%) after 1385.7 sec
1405.54/1405.59	c [picomus] extracted core 43 of size 63433 out of 168347 (38%) after 1405.6 sec
1425.84/1425.81	c [picomus] extracted core 44 of size 63300 out of 168347 (38%) after 1425.9 sec
1474.73/1474.72	c [picomus] extracted core 45 of size 63233 out of 168347 (38%) after 1474.8 sec
1513.34/1513.30	c [picomus] extracted core 46 of size 63191 out of 168347 (38%) after 1513.3 sec
1540.25/1540.21	c [picomus] extracted core 47 of size 63107 out of 168347 (37%) after 1540.3 sec
1583.95/1583.95	c [picomus] extracted core 48 of size 63070 out of 168347 (37%) after 1584.0 sec
1629.75/1629.78	c [picomus] extracted core 49 of size 63026 out of 168347 (37%) after 1629.8 sec
1656.24/1656.20	c [picomus] extracted core 50 of size 62958 out of 168347 (37%) after 1656.3 sec
1683.05/1683.00	c [picomus] extracted core 51 of size 62875 out of 168347 (37%) after 1683.1 sec
1728.64/1728.62	c [picomus] extracted core 52 of size 62822 out of 168347 (37%) after 1728.7 sec
1767.15/1767.14	c [picomus] extracted core 53 of size 62805 out of 168347 (37%) after 1767.2 sec

Verifier Data 

ERROR: no value line found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 988)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3493730-1307802650/watcher-3493730-1307802650 -o /tmp/evaluation-result-3493730-1307802650/solver-3493730-1307802650 -C 1800 -W 1850 -M 7680 picomus HOME/instance-3493730-1307802650.cnf 

running on 2 cores: 1,3

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1850 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.92 3.97 3.77 4/189 24849
/proc/meminfo: memFree=30802852/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=3164 CPUtime=0 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 280 0 0 0 0 0 0 0 25 0 1 0 970495719 3239936 235 33554432000 134512640 135072097 4288118368 18446744073709551615 134684699 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 791 235 53 137 0 652 0

[startup+0.0327061 s]
/proc/loadavg: 3.92 3.97 3.77 4/189 24849
/proc/meminfo: memFree=30802852/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=4748 CPUtime=0.03 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 954 0 0 0 3 0 0 0 25 0 1 0 970495719 4861952 909 33554432000 134512640 135072097 4288118368 18446744073709551615 134705502 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 1187 910 53 137 0 1048 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 4748

[startup+0.100691 s]
/proc/loadavg: 3.92 3.97 3.77 4/189 24849
/proc/meminfo: memFree=30802852/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=15760 CPUtime=0.09 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 3746 0 0 0 9 0 0 0 25 0 1 0 970495719 16138240 3677 33554432000 134512640 135072097 4288118368 18446744073709551615 134575537 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 3940 3677 68 137 0 3801 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 15760

[startup+0.300668 s]
/proc/loadavg: 3.92 3.97 3.77 4/189 24849
/proc/meminfo: memFree=30802852/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=16108 CPUtime=0.29 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 3836 0 0 0 29 0 0 0 25 0 1 0 970495719 16494592 3767 33554432000 134512640 135072097 4288118368 18446744073709551615 134543512 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 4027 3767 69 137 0 3888 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 16108

[startup+0.700578 s]
/proc/loadavg: 3.92 3.97 3.77 4/189 24849
/proc/meminfo: memFree=30802852/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=16372 CPUtime=0.69 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 3892 0 0 0 69 0 0 0 25 0 1 0 970495719 16764928 3823 33554432000 134512640 135072097 4288118368 18446744073709551615 134543547 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 4093 3823 69 137 0 3954 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 16372

[startup+1.50043 s]
/proc/loadavg: 3.92 3.97 3.77 5/190 24850
/proc/meminfo: memFree=30787752/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=16900 CPUtime=1.49 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 4017 0 0 0 149 0 0 0 25 0 1 0 970495719 17305600 3948 33554432000 134512640 135072097 4288118368 18446744073709551615 134557343 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 4225 3949 69 137 0 4086 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 16900

[startup+3.10113 s]
/proc/loadavg: 3.92 3.97 3.77 5/190 24850
/proc/meminfo: memFree=30786512/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=18220 CPUtime=3.09 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 4326 0 0 0 309 0 0 0 25 0 1 0 970495719 18657280 4257 33554432000 134512640 135072097 4288118368 18446744073709551615 134537972 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 4555 4257 69 137 0 4416 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 18220

[startup+6.30052 s]
/proc/loadavg: 3.93 3.97 3.77 5/190 24850
/proc/meminfo: memFree=30783996/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=20332 CPUtime=6.29 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 4871 0 0 0 628 1 0 0 25 0 1 0 970495719 20819968 4802 33554432000 134512640 135072097 4288118368 18446744073709551615 134558187 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 5083 4802 69 137 0 4944 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 20332

[startup+12.7003 s]
/proc/loadavg: 3.93 3.97 3.77 5/190 24851
/proc/meminfo: memFree=30778204/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=26264 CPUtime=12.69 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 6322 0 0 0 1268 1 0 0 25 0 1 0 970495719 26894336 6253 33554432000 134512640 135072097 4288118368 18446744073709551615 134543606 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 6566 6253 69 137 0 6427 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 26264

[startup+25.5009 s]
/proc/loadavg: 3.88 3.95 3.77 5/190 24928
/proc/meminfo: memFree=31105424/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=38524 CPUtime=25.49 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 9345 0 0 0 2547 2 0 0 25 0 1 0 970495719 39448576 9276 33554432000 134512640 135072097 4288118368 18446744073709551615 134543804 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 9631 9276 69 137 0 9492 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 38524

[startup+51.1011 s]
/proc/loadavg: 3.92 3.96 3.77 5/190 24929
/proc/meminfo: memFree=31067660/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=67784 CPUtime=51.09 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 16539 0 0 0 5105 4 0 0 25 0 1 0 970495719 69410816 16459 33554432000 134512640 135072097 4288118368 18446744073709551615 134543446 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 16946 16459 69 137 0 16807 0
Current children cumulated CPU time (s) 51.09
Current children cumulated vsize (KiB) 67784

[startup+102.306 s]
/proc/loadavg: 3.96 3.96 3.78 5/190 24930
/proc/meminfo: memFree=31054252/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=74180 CPUtime=102.3 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 34179 0 0 0 10221 9 0 0 25 0 1 0 970495719 75960320 18104 33554432000 134512640 135072097 4288118368 18446744073709551615 134543623 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 18545 18104 76 137 0 18406 0
Current children cumulated CPU time (s) 102.3
Current children cumulated vsize (KiB) 74180

[startup+162.301 s]
/proc/loadavg: 3.98 3.97 3.80 5/190 25047
/proc/meminfo: memFree=31201360/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=34492 CPUtime=162.29 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 59684 0 0 0 16216 13 0 0 25 0 1 0 970495719 35319808 8320 33554432000 134512640 135072097 4288118368 18446744073709551615 134557156 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 8623 8320 76 137 0 8484 0
Current children cumulated CPU time (s) 162.29
Current children cumulated vsize (KiB) 34492

[startup+222.301 s]
/proc/loadavg: 4.08 4.00 3.82 5/190 25048
/proc/meminfo: memFree=31181816/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=46784 CPUtime=222.3 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 85827 0 0 0 22212 18 0 0 25 0 1 0 970495719 47906816 11444 33554432000 134512640 135072097 4288118368 18446744073709551615 134520125 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 11696 11444 76 137 0 11557 0
Current children cumulated CPU time (s) 222.3
Current children cumulated vsize (KiB) 46784

[startup+282.3 s]
/proc/loadavg: 4.03 4.00 3.82 5/190 25050
/proc/meminfo: memFree=30917156/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=36168 CPUtime=282.3 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 111923 0 0 0 28207 23 0 0 25 0 1 0 970495719 37036032 8726 33554432000 134512640 135072097 4288118368 18446744073709551615 134518217 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 9042 8726 76 137 0 8903 0
Current children cumulated CPU time (s) 282.3
Current children cumulated vsize (KiB) 36168

[startup+342.301 s]
/proc/loadavg: 4.01 4.00 3.83 5/190 25051
/proc/meminfo: memFree=30891408/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=37512 CPUtime=342.3 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 138202 0 0 0 34201 29 0 0 25 0 1 0 970495719 38412288 9088 33554432000 134512640 135072097 4288118368 18446744073709551615 134520177 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 9378 9088 76 137 0 9239 0
Current children cumulated CPU time (s) 342.3
Current children cumulated vsize (KiB) 37512

[startup+402.301 s]
/proc/loadavg: 4.12 4.03 3.85 5/190 25053
/proc/meminfo: memFree=30900156/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=22972 CPUtime=402.31 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 164771 0 0 0 40196 35 0 0 25 0 1 0 970495719 23523328 5484 33554432000 134512640 135072097 4288118368 18446744073709551615 134520211 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 5743 5484 76 137 0 5604 0
Current children cumulated CPU time (s) 402.31
Current children cumulated vsize (KiB) 22972

[startup+462.3 s]
/proc/loadavg: 4.04 4.02 3.85 5/190 25055
/proc/meminfo: memFree=30880636/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=21276 CPUtime=462.31 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 190794 0 0 0 46191 40 0 0 25 0 1 0 970495719 21786624 5042 33554432000 134512640 135072097 4288118368 18446744073709551615 134543424 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 5319 5042 76 137 0 5180 0
Current children cumulated CPU time (s) 462.31
Current children cumulated vsize (KiB) 21276

[startup+522.301 s]
/proc/loadavg: 4.01 4.02 3.86 5/190 25059
/proc/meminfo: memFree=30871696/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=31164 CPUtime=522.3 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 217707 0 0 0 52183 47 0 0 25 0 1 0 970495719 31911936 7464 33554432000 134512640 135072097 4288118368 18446744073709551615 134520266 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 7791 7464 76 137 0 7652 0
Current children cumulated CPU time (s) 522.3
Current children cumulated vsize (KiB) 31164

[startup+582.301 s]
/proc/loadavg: 4.05 4.03 3.87 5/190 25061
/proc/meminfo: memFree=30831132/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=42344 CPUtime=582.31 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 244237 0 0 0 58178 53 0 0 25 0 1 0 970495719 43360256 10323 33554432000 134512640 135072097 4288118368 18446744073709551615 134580722 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 10586 10323 76 137 0 10447 0
Current children cumulated CPU time (s) 582.31
Current children cumulated vsize (KiB) 42344

[startup+642.3 s]

################
# More data... #
################

[startup+822.3 s]
/proc/loadavg: 4.01 4.02 3.90 5/190 25067
/proc/meminfo: memFree=30802292/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=23128 CPUtime=822.32 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 347522 0 0 0 82154 78 0 0 25 0 1 0 970495719 23683072 5507 33554432000 134512640 135072097 4288118368 18446744073709551615 134557294 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 5782 5507 76 137 0 5643 0
Current children cumulated CPU time (s) 822.32
Current children cumulated vsize (KiB) 23128

[startup+882.301 s]
/proc/loadavg: 4.07 4.03 3.91 5/190 25069
/proc/meminfo: memFree=30765148/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=59912 CPUtime=882.32 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 374421 0 0 0 88149 83 0 0 25 0 1 0 970495719 61349888 14541 33554432000 134512640 135072097 4288118368 18446744073709551615 134558531 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 14978 14541 76 137 0 14839 0
Current children cumulated CPU time (s) 882.32
Current children cumulated vsize (KiB) 59912

[startup+942.301 s]
/proc/loadavg: 4.06 4.04 3.91 5/190 25070
/proc/meminfo: memFree=30767440/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=58476 CPUtime=942.32 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 401402 0 0 0 94143 89 0 0 25 0 1 0 970495719 59879424 14178 33554432000 134512640 135072097 4288118368 18446744073709551615 134543824 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 14619 14178 76 137 0 14480 0
Current children cumulated CPU time (s) 942.32
Current children cumulated vsize (KiB) 58476

[startup+1002.3 s]
/proc/loadavg: 4.02 4.03 3.91 5/190 25072
/proc/meminfo: memFree=30756000/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=38312 CPUtime=1002.32 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 428076 0 0 0 100136 96 0 0 25 0 1 0 970495719 39231488 9267 33554432000 134512640 135072097 4288118368 18446744073709551615 134558206 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 9578 9267 76 137 0 9439 0
Current children cumulated CPU time (s) 1002.32
Current children cumulated vsize (KiB) 38312

[startup+1062.3 s]
/proc/loadavg: 4.01 4.02 3.91 5/190 25074
/proc/meminfo: memFree=30750964/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=44304 CPUtime=1062.33 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 453242 0 0 0 106131 102 0 0 25 0 1 0 970495719 45367296 10829 33554432000 134512640 135072097 4288118368 18446744073709551615 134543471 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 11076 10829 76 137 0 10937 0
Current children cumulated CPU time (s) 1062.33
Current children cumulated vsize (KiB) 44304

[startup+1122.3 s]
/proc/loadavg: 4.00 4.01 3.91 5/190 25075
/proc/meminfo: memFree=30768140/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=25404 CPUtime=1122.33 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 478613 0 0 0 112124 109 0 0 25 0 1 0 970495719 26013696 6116 33554432000 134512640 135072097 4288118368 18446744073709551615 134543400 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 6351 6117 76 137 0 6212 0
Current children cumulated CPU time (s) 1122.33
Current children cumulated vsize (KiB) 25404

[startup+1182.3 s]
/proc/loadavg: 4.00 4.01 3.91 5/190 25077
/proc/meminfo: memFree=30716276/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=78740 CPUtime=1182.33 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 505299 0 0 0 118119 114 0 0 25 0 1 0 970495719 80629760 19375 33554432000 134512640 135072097 4288118368 18446744073709551615 134556443 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 19685 19375 76 137 0 19546 0
Current children cumulated CPU time (s) 1182.33
Current children cumulated vsize (KiB) 78740

[startup+1242.3 s]
/proc/loadavg: 4.00 4.00 3.91 5/190 25078
/proc/meminfo: memFree=30754296/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=39360 CPUtime=1242.33 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 531819 0 0 0 124114 119 0 0 25 0 1 0 970495719 40304640 9551 33554432000 134512640 135072097 4288118368 18446744073709551615 134543804 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 9840 9551 76 137 0 9701 0
Current children cumulated CPU time (s) 1242.33
Current children cumulated vsize (KiB) 39360

[startup+1302.3 s]
/proc/loadavg: 4.04 4.01 3.92 5/190 25080
/proc/meminfo: memFree=30760080/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=32472 CPUtime=1302.34 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 558030 0 0 0 130110 124 0 0 25 0 1 0 970495719 33251328 7801 33554432000 134512640 135072097 4288118368 18446744073709551615 134548653 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 8118 7801 76 137 0 7979 0
Current children cumulated CPU time (s) 1302.34
Current children cumulated vsize (KiB) 32472

[startup+1362.3 s]
/proc/loadavg: 4.01 4.01 3.92 5/190 25082
/proc/meminfo: memFree=30771188/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=22196 CPUtime=1362.34 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 583927 0 0 0 136104 130 0 0 25 0 1 0 970495719 22728704 5309 33554432000 134512640 135072097 4288118368 18446744073709551615 134543775 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 5549 5309 76 137 0 5410 0
Current children cumulated CPU time (s) 1362.34
Current children cumulated vsize (KiB) 22196

[startup+1422.31 s]
/proc/loadavg: 4.00 4.00 3.92 5/190 25083
/proc/meminfo: memFree=30755992/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=35960 CPUtime=1422.34 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 609494 0 0 0 142099 135 0 0 25 0 1 0 970495719 36823040 8697 33554432000 134512640 135072097 4288118368 18446744073709551615 134520136 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 8990 8697 76 137 0 8851 0
Current children cumulated CPU time (s) 1422.34
Current children cumulated vsize (KiB) 35960

[startup+1482.3 s]
/proc/loadavg: 4.06 4.01 3.93 5/190 25085
/proc/meminfo: memFree=30774180/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=18608 CPUtime=1482.34 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 636242 0 0 0 148092 142 0 0 25 0 1 0 970495719 19054592 4378 33554432000 134512640 135072097 4288118368 18446744073709551615 134543834 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 4652 4378 76 137 0 4513 0
Current children cumulated CPU time (s) 1482.34
Current children cumulated vsize (KiB) 18608

[startup+1542.3 s]
/proc/loadavg: 4.02 4.01 3.93 5/190 25086
/proc/meminfo: memFree=30778228/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=13640 CPUtime=1542.34 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 662212 0 0 0 154085 149 0 0 25 0 1 0 970495719 13967360 3192 33554432000 134512640 135072097 4288118368 18446744073709551615 134543541 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 3410 3192 76 137 0 3271 0
Current children cumulated CPU time (s) 1542.34
Current children cumulated vsize (KiB) 13640

[startup+1602.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/190 25088
/proc/meminfo: memFree=30754988/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=38772 CPUtime=1602.35 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 686963 0 0 0 160081 154 0 0 25 0 1 0 970495719 39702528 9399 33554432000 134512640 135072097 4288118368 18446744073709551615 134558408 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 9693 9399 76 137 0 9554 0
Current children cumulated CPU time (s) 1602.35
Current children cumulated vsize (KiB) 38772

[startup+1662.3 s]
/proc/loadavg: 3.92 3.98 3.92 4/184 25115
/proc/meminfo: memFree=30905868/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=17588 CPUtime=1662.35 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 713880 0 0 0 166074 161 0 0 25 0 1 0 970495719 18010112 4189 33554432000 134512640 135072097 4288118368 18446744073709551615 134556761 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 4397 4189 76 137 0 4258 0
Current children cumulated CPU time (s) 1662.35
Current children cumulated vsize (KiB) 17588

[startup+1722.3 s]
/proc/loadavg: 3.34 3.80 3.86 4/184 25116
/proc/meminfo: memFree=30849732/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=75756 CPUtime=1722.35 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 739871 0 0 0 172068 167 0 0 25 0 1 0 970495719 77574144 18607 33554432000 134512640 135072097 4288118368 18446744073709551615 134543952 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 18939 18607 76 137 0 18800 0
Current children cumulated CPU time (s) 1722.35
Current children cumulated vsize (KiB) 75756

[startup+1782.3 s]
/proc/loadavg: 3.12 3.65 3.80 4/184 25118
/proc/meminfo: memFree=30889352/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=35104 CPUtime=1782.35 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 766687 0 0 0 178060 175 0 0 25 0 1 0 970495719 35946496 8442 33554432000 134512640 135072097 4288118368 18446744073709551615 134548676 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 8776 8442 76 137 0 8637 0
Current children cumulated CPU time (s) 1782.35
Current children cumulated vsize (KiB) 35104



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800 s]
/proc/loadavg: 3.09 3.61 3.79 4/184 25118
/proc/meminfo: memFree=30857216/32950928 swapFree=67056212/67111528
[pid=24849] ppid=24847 vsize=66620 CPUtime=1800.05 cores=1,3
/proc/24849/stat : 24849 (picomus) R 24847 24849 24520 0 -1 4202496 774498 0 0 0 179829 176 0 0 25 0 1 0 970495719 68218880 16253 33554432000 134512640 135072097 4288118368 18446744073709551615 134684699 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/24849/statm: 16655 16253 76 137 0 16516 0
Current children cumulated CPU time (s) 1800.05
Current children cumulated vsize (KiB) 66620

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1800.01
CPU time (s): 1800.07
CPU user time (s): 1798.3
CPU system time (s): 1.77173
CPU usage (%): 100.003
Max. virtual memory (cumulated for all children) (KiB): 102792

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1798.3
system time used= 1.77173
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 774498
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4
involuntary context switches= 1099

runsolver used 2.94555 second user time and 6.88895 second system time

The end

Launcher Data 

Begin job on node115 at 2011-06-11 16:30:51
IDJOB=3493730
IDBENCH=82490
IDSOLVER=1974
FILE ID=node115/3493730-1307802650
RUNJOBID= node115-1307800705-24540
PBS_JOBID= 13551062
Free space on /tmp= 74080 MiB

SOLVER NAME= PicoMUS 941
BENCH NAME= SAT11/application/jarvisalo/HWMCC10-combination-timeframe-expansion/hwmcc10-timeframe-expansion-k50-viscoherencep3-tseitin.cnf
COMMAND LINE= picomus BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3493730-1307802650/watcher-3493730-1307802650 -o /tmp/evaluation-result-3493730-1307802650/solver-3493730-1307802650 -C 1800 -W 1850 -M 7680  picomus HOME/instance-3493730-1307802650.cnf

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= e6bb96c7140e2a8d07499de24fcd1583
RANDOM SEED=816334429

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30803396 kB
Buffers:        292292 kB
Cached:        1091356 kB
SwapCached:       9476 kB
Active:        1038760 kB
Inactive:       915624 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30803396 kB
SwapTotal:    67111528 kB
SwapFree:     67056212 kB
Dirty:            3544 kB
Writeback:           0 kB
AnonPages:      567724 kB
Mapped:          14156 kB
Slab:           126052 kB
PageTables:       6344 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   836252 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74120 MiB
End job on node115 at 2011-06-11 17:00:53