Trace number 3387627

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
minisat hack: minisat_psm 2011-04-01? (TO) 5000.04 5000.12

General information on the benchmark

NameSAT-Race-2010-CNF/hardware-verification/
velev/11pipe_11_ooo.cnf
MD5SUMf603f11fbfe855df381282b7bce010ea
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark2200.85
Satisfiable
(Un)Satisfiability was proved
Number of variables88289
Number of clauses4187694
Sum of the clauses size12410354
Maximum clause length341
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 23770000
Number of clauses of size 3347602
Number of clauses of size 41622
Number of clauses of size 51351
Number of clauses of size over 567118

Solver Data 

0.00/0.00	c Parsing...
1.39/1.50	c ==============================================================================
1.39/1.50	c |           |     ORIGINAL     |              LEARNT              |          |
1.39/1.50	c | Conflicts | Clauses Literals |   Limit Clauses Literals  Lit/Cl | Progress |
1.39/1.50	c ==============================================================================
1.49/1.50	c |         0 | 4187693 12410353 | 1256307       0        0     nan |  0.000 % |
3.08/11.09	c |         0 | 4155136 12333878 |      --       0       --      -- |     --   | -32445/-76228
3.08/11.09	c ==============================================================================
3.08/11.09	c Result  :   #vars: 77804   #clauses: 4155136   #literals: 12333878
3.08/11.09	c CPU time:   10.7724 s
3.08/11.09	c ==============================================================================

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3387627-1305607240/watcher-3387627-1305607240 -o /tmp/evaluation-result-3387627-1305607240/solver-3387627-1305607240 -C 5000 -W 5100 -M 15500 ./esg_static.sh HOME/instance-3387627-1305607240.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.15 3.63 3.86 4/436 30959
/proc/meminfo: memFree=13229540/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 327 0 0 0 0 0 0 0 23 0 1 0 672323853 8908800 249 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/30959/statm: 2175 249 209 178 0 68 0

[startup+0.092267 s]
/proc/loadavg: 3.15 3.63 3.86 4/436 30959
/proc/meminfo: memFree=13229540/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 327 0 0 0 0 0 0 0 23 0 1 0 672323853 8908800 249 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/30959/statm: 2175 249 209 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.100261 s]
/proc/loadavg: 3.15 3.63 3.86 4/436 30959
/proc/meminfo: memFree=13229540/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 327 0 0 0 0 0 0 0 23 0 1 0 672323853 8908800 249 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/30959/statm: 2175 249 209 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.300226 s]
/proc/loadavg: 3.15 3.63 3.86 4/436 30959
/proc/meminfo: memFree=13229540/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 327 0 0 0 0 0 0 0 23 0 1 0 672323853 8908800 249 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/30959/statm: 2175 249 209 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.700161 s]
/proc/loadavg: 3.15 3.63 3.86 4/436 30959
/proc/meminfo: memFree=13229540/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 327 0 0 0 0 0 0 0 23 0 1 0 672323853 8908800 249 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/30959/statm: 2175 249 209 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50102 s]
/proc/loadavg: 3.15 3.63 3.86 4/438 30961
/proc/meminfo: memFree=13086760/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 327 0 0 0 0 0 0 0 23 0 1 0 672323853 8908800 249 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/30959/statm: 2175 249 209 178 0 68 0
[pid=30960] ppid=30959 vsize=244536 CPUtime=1.49 cores=1,3,5,7
/proc/30960/stat : 30960 (SatELite_releas) R 30959 30959 28742 0 -1 4202496 59761 0 0 0 130 19 0 0 25 0 1 0 672323853 250404864 59716 33554432000 134512640 135144242 4288312128 18446744073709551615 134575972 0 0 4096 3 0 0 0 17 1 0 0 0
/proc/30960/statm: 61134 59731 70 155 0 60977 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 253236

[startup+3.10076 s]
/proc/loadavg: 3.15 3.63 3.86 4/438 30961
/proc/meminfo: memFree=12944408/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 327 0 0 0 0 0 0 0 23 0 1 0 672323853 8908800 249 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/30959/statm: 2175 249 209 178 0 68 0
[pid=30960] ppid=30959 vsize=320756 CPUtime=3.08 cores=1,3,5,7
/proc/30960/stat : 30960 (SatELite_releas) R 30959 30959 28742 0 -1 4202496 79159 0 0 0 287 21 0 0 25 0 1 0 672323853 328454144 71323 33554432000 134512640 135144242 4288312128 18446744073709551615 134554984 0 0 4096 3 0 0 0 17 1 0 0 0
/proc/30960/statm: 80189 71323 77 155 0 80032 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 329456

[startup+27.2435 s]
/proc/loadavg: 4.06 3.80 3.91 4/438 30963
/proc/meminfo: memFree=12918160/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=262168 CPUtime=14.11 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 110639 0 0 0 1395 16 0 0 25 0 1 0 672325164 268460032 53996 33554432000 4194304 4256717 140736925662512 18446744073709551615 4213416 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 65542 53996 259 16 0 62184 0
Current children cumulated CPU time (s) 27.14
Current children cumulated vsize (KiB) 270868

[startup+27.3186 s]
/proc/loadavg: 4.06 3.80 3.91 4/438 30963
/proc/meminfo: memFree=12918160/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=262168 CPUtime=14.16 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 110658 0 0 0 1400 16 0 0 25 0 1 0 672325164 268460032 54015 33554432000 4194304 4256717 140736925662512 18446744073709551615 4213501 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 65542 54015 259 16 0 62184 0
Current children cumulated CPU time (s) 27.19
Current children cumulated vsize (KiB) 270868

[startup+27.4006 s]
/proc/loadavg: 4.06 3.80 3.91 4/438 30963
/proc/meminfo: memFree=12918160/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=262300 CPUtime=14.25 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 110723 0 0 0 1409 16 0 0 25 0 1 0 672325164 268595200 54080 33554432000 4194304 4256717 140736925662512 18446744073709551615 4210920 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 65575 54080 259 16 0 62217 0
Current children cumulated CPU time (s) 27.28
Current children cumulated vsize (KiB) 271000

[startup+51.1005 s]
/proc/loadavg: 4.62 3.95 3.95 4/438 30964
/proc/meminfo: memFree=12910744/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=265992 CPUtime=37.95 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 246642 0 0 0 3764 31 0 0 25 0 1 0 672325164 272375808 56442 33554432000 4194304 4256717 140736925662512 18446744073709551615 4233419 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 66498 56442 259 16 0 63140 0
Current children cumulated CPU time (s) 50.98
Current children cumulated vsize (KiB) 274692

[startup+102.301 s]
/proc/loadavg: 3.70 3.80 3.90 4/438 30965
/proc/meminfo: memFree=12903728/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=271348 CPUtime=89.12 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 482167 0 0 0 8852 60 0 0 25 0 1 0 672325164 277860352 61536 33554432000 4194304 4256717 140736925662512 18446744073709551615 4213972 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 67837 61536 259 16 0 64479 0
Current children cumulated CPU time (s) 102.15
Current children cumulated vsize (KiB) 280048

[startup+162.3 s]
/proc/loadavg: 3.26 3.65 3.84 4/438 30966
/proc/meminfo: memFree=12882144/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=275048 CPUtime=149.12 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 826045 0 0 0 14808 104 0 0 25 0 1 0 672325164 281649152 64403 33554432000 4194304 4256717 140736925662512 18446744073709551615 4213584 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 68762 64403 259 16 0 65404 0
Current children cumulated CPU time (s) 162.15
Current children cumulated vsize (KiB) 283748

[startup+222.301 s]
/proc/loadavg: 3.21 3.56 3.79 4/438 30968
/proc/meminfo: memFree=12883864/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=278144 CPUtime=209.12 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 1118929 0 0 0 20769 143 0 0 25 0 1 0 672325164 284819456 64604 33554432000 4194304 4256717 140736925662512 18446744073709551615 4233419 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 69536 64604 259 16 0 66178 0
Current children cumulated CPU time (s) 222.15
Current children cumulated vsize (KiB) 286844

[startup+282.301 s]
/proc/loadavg: 3.07 3.45 3.74 4/438 30970
/proc/meminfo: memFree=12891044/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=282088 CPUtime=269.13 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 1473945 0 0 0 26721 192 0 0 25 0 1 0 672325164 288858112 64338 33554432000 4194304 4256717 140736925662512 18446744073709551615 4213501 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 70522 64338 259 16 0 67164 0
Current children cumulated CPU time (s) 282.16
Current children cumulated vsize (KiB) 290788

[startup+342.3 s]
/proc/loadavg: 3.02 3.37 3.69 4/438 30971
/proc/meminfo: memFree=12849120/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=365876 CPUtime=329.12 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 1733546 0 0 0 32687 225 0 0 25 0 1 0 672325164 374657024 74278 33554432000 4194304 4256717 140736925662512 18446744073709551615 4210789 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 91469 74278 259 16 0 88111 0
Current children cumulated CPU time (s) 342.15
Current children cumulated vsize (KiB) 374576

[startup+402.301 s]
/proc/loadavg: 3.01 3.30 3.65 4/438 30973
/proc/meminfo: memFree=12871424/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7

################
# More data... #
################

/proc/30962/statm: 112256 101547 259 16 0 108898 0
Current children cumulated CPU time (s) 4362.24
Current children cumulated vsize (KiB) 457724

[startup+4422.3 s]
/proc/loadavg: 1.01 1.01 1.05 2/430 31110
/proc/meminfo: memFree=12947928/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=449208 CPUtime=4409.19 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 15354524 0 0 0 438862 2057 0 0 25 0 1 0 672325164 459988992 102651 33554432000 4194304 4256717 140736925662512 18446744073709551615 4213369 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 112302 102651 259 16 0 108944 0
Current children cumulated CPU time (s) 4422.22
Current children cumulated vsize (KiB) 457908

[startup+4482.32 s]
/proc/loadavg: 1.00 1.01 1.05 2/430 31112
/proc/meminfo: memFree=12935644/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=449580 CPUtime=4469.21 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 15488096 0 0 0 444847 2074 0 0 25 0 1 0 672325164 460369920 105430 33554432000 4194304 4256717 140736925662512 18446744073709551615 4213192 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 112395 105430 259 16 0 109037 0
Current children cumulated CPU time (s) 4482.24
Current children cumulated vsize (KiB) 458280

[startup+4542.3 s]
/proc/loadavg: 1.00 1.00 1.04 2/430 31113
/proc/meminfo: memFree=12956596/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=450216 CPUtime=4529.2 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 15615404 0 0 0 450828 2092 0 0 25 0 1 0 672325164 461021184 100233 33554432000 4194304 4256717 140736925662512 18446744073709551615 4211816 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 112554 100233 259 16 0 109196 0
Current children cumulated CPU time (s) 4542.23
Current children cumulated vsize (KiB) 458916

[startup+4602.3 s]
/proc/loadavg: 1.00 1.00 1.04 2/430 31115
/proc/meminfo: memFree=12912196/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=582800 CPUtime=4589.2 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 15690668 0 0 0 456815 2105 0 0 25 0 1 0 672325164 596787200 111302 33554432000 4194304 4256717 140736925662512 18446744073709551615 4213972 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 145700 111302 259 16 0 142342 0
Current children cumulated CPU time (s) 4602.23
Current children cumulated vsize (KiB) 591500

[startup+4662.3 s]
/proc/loadavg: 1.00 1.00 1.03 2/430 31116
/proc/meminfo: memFree=12956204/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=451124 CPUtime=4649.2 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 15821496 0 0 0 462794 2126 0 0 25 0 1 0 672325164 461950976 100754 33554432000 4194304 4256717 140736925662512 18446744073709551615 4214122 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 112781 100754 259 16 0 109423 0
Current children cumulated CPU time (s) 4662.23
Current children cumulated vsize (KiB) 459824

[startup+4722.3 s]
/proc/loadavg: 1.00 1.00 1.02 2/430 31118
/proc/meminfo: memFree=12931272/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=451372 CPUtime=4709.2 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 15899142 0 0 0 468782 2138 0 0 25 0 1 0 672325164 462204928 106420 33554432000 4194304 4256717 140736925662512 18446744073709551615 4210891 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 112843 106420 259 16 0 109485 0
Current children cumulated CPU time (s) 4722.23
Current children cumulated vsize (KiB) 460072

[startup+4782.3 s]
/proc/loadavg: 1.00 1.00 1.02 2/430 31120
/proc/meminfo: memFree=12885016/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=583932 CPUtime=4769.2 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 15984487 0 0 0 474768 2152 0 0 25 0 1 0 672325164 597946368 118302 33554432000 4194304 4256717 140736925662512 18446744073709551615 4213452 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 145983 118302 259 16 0 142625 0
Current children cumulated CPU time (s) 4782.23
Current children cumulated vsize (KiB) 592632

[startup+4842.3 s]
/proc/loadavg: 1.00 1.00 1.01 2/430 31130
/proc/meminfo: memFree=12956160/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=454340 CPUtime=4829.2 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 16178258 0 0 0 480743 2177 0 0 25 0 1 0 672325164 465244160 100320 33554432000 4194304 4256717 140736925662512 18446744073709551615 4209093 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 113585 100320 259 16 0 110227 0
Current children cumulated CPU time (s) 4842.23
Current children cumulated vsize (KiB) 463040

[startup+4902.3 s]
/proc/loadavg: 1.03 1.01 1.01 2/430 31132
/proc/meminfo: memFree=12959380/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=454340 CPUtime=4889.2 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 16300732 0 0 0 486726 2194 0 0 25 0 1 0 672325164 465244160 99734 33554432000 4194304 4256717 140736925662512 18446744073709551615 4209058 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 113585 99734 259 16 0 110227 0
Current children cumulated CPU time (s) 4902.23
Current children cumulated vsize (KiB) 463040

[startup+4962.3 s]
/proc/loadavg: 1.01 1.01 1.01 2/430 31133
/proc/meminfo: memFree=12968912/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=454340 CPUtime=4949.21 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 16430385 0 0 0 492709 2212 0 0 25 0 1 0 672325164 465244160 97174 33554432000 4194304 4256717 140736925662512 18446744073709551615 4210789 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 113585 97174 259 16 0 110227 0
Current children cumulated CPU time (s) 4962.24
Current children cumulated vsize (KiB) 463040



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5000.1 s]
/proc/loadavg: 1.00 1.00 1.00 2/430 31135
/proc/meminfo: memFree=12967664/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=455856 CPUtime=4987.01 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 16495873 0 0 0 496481 2220 0 0 25 0 1 0 672325164 466796544 97361 33554432000 4194304 4256717 140736925662512 18446744073709551615 4233419 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 113964 97361 259 16 0 110606 0
Current children cumulated CPU time (s) 5000.04
Current children cumulated vsize (KiB) 464556

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5000.1 s]
/proc/loadavg: 1.00 1.00 1.00 2/430 31135
/proc/meminfo: memFree=12967664/32951132 swapFree=67093824/67111528
[pid=30959] ppid=30957 vsize=8700 CPUtime=13.03 cores=1,3,5,7
/proc/30959/stat : 30959 (esg_static.sh) S 30957 30959 28742 0 -1 4202496 353 102770 0 0 0 0 1265 38 18 0 1 0 672323853 8908800 251 33554432000 4194304 4922060 140737022783168 18446744073709551615 274664627333 0 65536 4100 65538 18446744071562232103 0 0 17 7 0 0 0
/proc/30959/statm: 2175 251 209 178 0 68 0
[pid=30962] ppid=30959 vsize=455856 CPUtime=4987.01 cores=1,3,5,7
/proc/30962/stat : 30962 (minisat_static) R 30959 30959 28742 0 -1 4202496 16495873 0 0 0 496481 2220 0 0 25 0 1 0 672325164 466796544 97361 33554432000 4194304 4256717 140736925662512 18446744073709551615 4233419 0 0 4096 8388610 0 0 0 17 3 0 0 0
/proc/30962/statm: 113964 97361 259 16 0 110606 0
Current children cumulated CPU time (s) 5000.04
Current children cumulated vsize (KiB) 464556

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 30959 and gives
#  childrusage.ru_utime.tv_sec=12
#  childrusage.ru_utime.tv_usec=659075
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=383941
# CPU time returned by wait4() is 13.043
# while last known CPU time is 5000.04
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.12
CPU time (s): 5000.04
CPU user time (s): 4977.46
CPU system time (s): 22.58
CPU usage (%): 99.9983
Max. virtual memory (cumulated for all children) (KiB): 806916

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 12.6591
system time used= 0.383941
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 103123
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8
involuntary context switches= 89

runsolver used 19.963 second user time and 50.0684 second system time

The end

Launcher Data 

Begin job on node131 at 2011-05-17 06:40:42
IDJOB=3387627
IDBENCH=88163
IDSOLVER=1874
FILE ID=node131/3387627-1305607240
RUNJOBID= node131-1305601334-30554
PBS_JOBID= 13350347
Free space on /tmp= 73552 MiB

SOLVER NAME= minisat hack: minisat_psm 2011-04-01
BENCH NAME= SAT-Race-2010-CNF/hardware-verification/velev/11pipe_11_ooo.cnf
COMMAND LINE= ./esg_static.sh BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3387627-1305607240/watcher-3387627-1305607240 -o /tmp/evaluation-result-3387627-1305607240/solver-3387627-1305607240 -C 5000 -W 5100 -M 15500  ./esg_static.sh HOME/instance-3387627-1305607240.cnf

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f603f11fbfe855df381282b7bce010ea
RANDOM SEED=508275128

node131.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      13229572 kB
Buffers:        317176 kB
Cached:        4785536 kB
SwapCached:       1504 kB
Active:       15774800 kB
Inactive:      3647884 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      13229572 kB
SwapTotal:    67111528 kB
SwapFree:     67093824 kB
Dirty:           82488 kB
Writeback:           0 kB
AnonPages:    14317904 kB
Mapped:          38600 kB
Slab:           174720 kB
PageTables:      56356 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS: 95813360 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73472 MiB
End job on node131 at 2011-05-17 08:04:04