Trace number 3293676

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.32 1201.71

General information on the benchmark

NameSAT-Race-2010-CNF/hardware-verification/
velev/2dlx_ca_bp_f_liveness.cnf
MD5SUM99cd8de7f06ce8c087a5c7fb299dfb62
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables202253
Number of clauses4313014
Sum of the clauses size12562188
Maximum clause length651
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 23935640
Number of clauses of size 3199653
Number of clauses of size 410363
Number of clauses of size 55846
Number of clauses of size over 5161511

Solver Data 

0.00/0.81	This is pico-portfolio, a naive parallel portfolio
0.00/0.81	Copyright (C) 2011 Olivier Roussel
0.00/0.81	Using 4 processing units
0.00/0.81	This portfolio uses the following solvers:
0.00/0.81	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.81	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.81	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.81	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.81	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.81	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.81	Starting clasp 1.3.6 on core(s) 5
0.00/0.81	Starting TNM 2009 on core(s) 7
0.00/0.81	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.81	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3293676-1304098012/watcher-3293676-1304098012 -o /tmp/evaluation-result-3293676-1304098012/solver-3293676-1304098012 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3293676-1304098012.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.67 5.00 5.22 1/171 25268
/proc/meminfo: memFree=15132408/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0

[startup+0.935368 s]
/proc/loadavg: 4.67 5.00 5.22 1/171 25268
/proc/meminfo: memFree=15132408/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.00036 s]
/proc/loadavg: 4.67 5.00 5.22 1/171 25268
/proc/meminfo: memFree=15132408/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.13533 s]
/proc/loadavg: 4.67 5.00 5.22 1/171 25268
/proc/meminfo: memFree=15132408/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.20032 s]
/proc/loadavg: 4.67 5.00 5.22 1/171 25268
/proc/meminfo: memFree=15132408/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50027 s]
/proc/loadavg: 4.67 5.00 5.22 1/171 25268
/proc/meminfo: memFree=15132408/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+3.10699 s]
/proc/loadavg: 4.67 5.00 5.22 6/177 25274
/proc/meminfo: memFree=14344112/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=455196 CPUtime=3.09 cores=1
/proc/25269/stat : 25269 (cryptominisat) R 25268 25268 24128 0 -1 4202496 99433 0 0 0 292 17 0 0 25 0 1 0 600031914 466120704 98686 33554432000 4194304 4587354 140733367454272 18446744073709551615 216734875403 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25269/statm: 113799 98686 308 96 0 110361 0
[pid=25270] ppid=25268 vsize=140232 CPUtime=3.09 cores=5
/proc/25270/stat : 25270 (clasp) R 25268 25268 24128 0 -1 4202496 37374 0 0 0 300 9 0 0 25 0 1 0 600031914 143597568 31781 33554432000 4194304 4785740 140736425602576 18446744073709551615 4545881 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25270/statm: 35058 31781 336 145 0 32103 0
[pid=25271] ppid=25268 vsize=477080 CPUtime=1.12 cores=7
/proc/25271/stat : 25271 (TNM) R 25268 25268 24128 0 -1 4202496 10371 0 0 0 109 3 0 0 25 0 1 0 600031914 488529920 10269 33554432000 134512640 134534564 4294838384 18446744073709551615 6190647 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25271/statm: 119270 10269 90 6 0 117936 0
[pid=25272] ppid=25268 vsize=98868 CPUtime=3.09 cores=3
/proc/25272/stat : 25272 (lingeling) R 25268 25268 24128 0 -1 4202496 31747 0 0 0 300 9 0 0 25 0 1 0 600031914 101240832 23899 33554432000 4194304 4352236 140734026021312 18446744073709551615 4248132 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25272/statm: 24717 23899 115 39 0 23797 0
[pid=25273] ppid=25268 vsize=187444 CPUtime=1.96 cores=7
/proc/25273/stat : 25273 (march_hi) R 25268 25268 24128 0 -1 4194304 44206 0 0 0 185 11 0 0 21 0 1 0 600031914 191942656 44101 33554432000 4194304 4331988 140735137827728 18446744073709551615 216734726745 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25273/statm: 46861 44101 113 34 0 45303 0
Current children cumulated CPU time (s) 12.35
Current children cumulated vsize (KiB) 1377144

[startup+6.30044 s]
/proc/loadavg: 4.78 5.02 5.22 6/177 25274
/proc/meminfo: memFree=14004848/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=509804 CPUtime=6.29 cores=1
/proc/25269/stat : 25269 (cryptominisat) R 25268 25268 24128 0 -1 4202496 113465 0 0 0 610 19 0 0 25 0 1 0 600031914 522039296 112718 33554432000 4194304 4587354 140733367454272 18446744073709551615 4357668 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25269/statm: 127451 112718 309 96 0 124013 0
[pid=25270] ppid=25268 vsize=149792 CPUtime=6.3 cores=5
/proc/25270/stat : 25270 (clasp) R 25268 25268 24128 0 -1 4202496 39700 0 0 0 620 10 0 0 25 0 1 0 600031914 153387008 34107 33554432000 4194304 4785740 140736425602576 18446744073709551615 4416773 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25270/statm: 37448 34107 336 145 0 34493 0
[pid=25271] ppid=25268 vsize=511004 CPUtime=2.72 cores=7
/proc/25271/stat : 25271 (TNM) R 25268 25268 24128 0 -1 4202496 24010 0 0 0 264 8 0 0 25 0 1 0 600031914 523268096 23908 33554432000 134512640 134534564 4294838384 18446744073709551615 6190659 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25271/statm: 127751 23908 90 6 0 126417 0
[pid=25272] ppid=25268 vsize=178240 CPUtime=6.29 cores=3
/proc/25272/stat : 25272 (lingeling) R 25268 25268 24128 0 -1 4202496 70571 0 0 0 616 13 0 0 25 0 1 0 600031914 182517760 43716 33554432000 4194304 4352236 140734026021312 18446744073709551615 4286618 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25272/statm: 44560 43716 121 39 0 43640 0
[pid=25273] ppid=25268 vsize=445064 CPUtime=3.53 cores=7
/proc/25273/stat : 25273 (march_hi) R 25268 25268 24128 0 -1 4194304 83257 0 0 0 335 18 0 0 25 0 1 0 600031914 455745536 67683 33554432000 4194304 4331988 140735137827728 18446744073709551615 216734836539 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25273/statm: 111266 67683 118 34 0 109708 0
Current children cumulated CPU time (s) 25.13
Current children cumulated vsize (KiB) 1812228

[startup+12.7013 s]
/proc/loadavg: 4.80 5.02 5.22 6/177 25274
/proc/meminfo: memFree=13470176/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=987208 CPUtime=12.69 cores=1
/proc/25269/stat : 25269 (cryptominisat) R 25268 25268 24128 0 -1 4202496 268291 0 0 0 1228 41 0 0 25 0 1 0 600031914 1010900992 192128 33554432000 4194304 4587354 140733367454272 18446744073709551615 4317864 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25269/statm: 246802 192128 320 96 0 243364 0
[pid=25270] ppid=25268 vsize=158280 CPUtime=12.69 cores=5
/proc/25270/stat : 25270 (clasp) R 25268 25268 24128 0 -1 4202496 41747 0 0 0 1259 10 0 0 25 0 1 0 600031914 162078720 36154 33554432000 4194304 4785740 140736425602576 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25270/statm: 39570 36154 336 145 0 36615 0
[pid=25271] ppid=25268 vsize=720184 CPUtime=5.92 cores=7
/proc/25271/stat : 25271 (TNM) R 25268 25268 24128 0 -1 4202496 85171 0 0 0 573 19 0 0 25 0 1 0 600031914 737468416 85068 33554432000 134512640 134534564 4294838384 18446744073709551615 134524672 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25271/statm: 180046 85068 91 6 0 178712 0
[pid=25272] ppid=25268 vsize=194920 CPUtime=12.69 cores=3
/proc/25272/stat : 25272 (lingeling) R 25268 25268 24128 0 -1 4202496 74751 0 0 0 1255 14 0 0 25 0 1 0 600031914 199598080 47896 33554432000 4194304 4352236 140734026021312 18446744073709551615 4208218 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25272/statm: 48730 47896 125 39 0 47810 0
[pid=25273] ppid=25268 vsize=566456 CPUtime=6.7 cores=7
/proc/25273/stat : 25273 (march_hi) R 25268 25268 24128 0 -1 4194304 149460 0 0 0 642 28 0 0 25 0 1 0 600031914 580050944 92531 33554432000 4194304 4331988 140735137827728 18446744073709551615 4252204 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25273/statm: 141614 92531 127 34 0 140056 0
Current children cumulated CPU time (s) 50.69
Current children cumulated vsize (KiB) 2645372

[startup+25.5011 s]
/proc/loadavg: 4.91 5.03 5.22 6/177 25275
/proc/meminfo: memFree=13341008/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=803656 CPUtime=25.49 cores=1
/proc/25269/stat : 25269 (cryptominisat) R 25268 25268 24128 0 -1 4202496 284420 0 0 0 2507 42 0 0 25 0 1 0 600031914 822943744 177658 33554432000 4194304 4587354 140733367454272 18446744073709551615 216734875281 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25269/statm: 200914 177658 327 96 0 197476 0
[pid=25270] ppid=25268 vsize=171960 CPUtime=25.49 cores=5
/proc/25270/stat : 25270 (clasp) R 25268 25268 24128 0 -1 4202496 45104 0 0 0 2537 12 0 0 25 0 1 0 600031914 176087040 39511 33554432000 4194304 4785740 140736425602576 18446744073709551615 4277472 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25270/statm: 42990 39511 336 145 0 40035 0
[pid=25271] ppid=25268 vsize=720184 CPUtime=12.26 cores=7
/proc/25271/stat : 25271 (TNM) R 25268 25268 24128 0 -1 4202496 85171 0 0 0 1207 19 0 0 25 0 1 0 600031914 737468416 85068 33554432000 134512640 134534564 4294838384 18446744073709551615 134524700 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25271/statm: 180046 85068 91 6 0 178712 0
[pid=25272] ppid=25268 vsize=199368 CPUtime=25.49 cores=3
/proc/25272/stat : 25272 (lingeling) R 25268 25268 24128 0 -1 4202496 75882 0 0 0 2535 14 0 0 25 0 1 0 600031914 204152832 49010 33554432000 4194304 4352236 140734026021312 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25272/statm: 49842 49010 127 39 0 48922 0
[pid=25273] ppid=25268 vsize=453096 CPUtime=13.09 cores=7
/proc/25273/stat : 25273 (march_hi) R 25268 25268 24128 0 -1 4202496 259973 0 0 0 1262 47 0 0 25 0 1 0 600031914 463970304 69705 33554432000 4194304 4331988 140735137827728 18446744073709551615 4251963 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25273/statm: 113274 69705 147 34 0 111716 0
Current children cumulated CPU time (s) 101.82
Current children cumulated vsize (KiB) 2366588

[startup+51.1006 s]
/proc/loadavg: 4.94 5.03 5.22 6/177 25275
/proc/meminfo: memFree=13377064/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=803656 CPUtime=51.09 cores=1
/proc/25269/stat : 25269 (cryptominisat) R 25268 25268 24128 0 -1 4202496 286227 0 0 0 5065 44 0 0 25 0 1 0 600031914 822943744 179456 33554432000 4194304 4587354 140733367454272 18446744073709551615 4238602 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25269/statm: 200914 179456 331 96 0 197476 0
[pid=25270] ppid=25268 vsize=211668 CPUtime=51.09 cores=5
/proc/25270/stat : 25270 (clasp) R 25268 25268 24128 0 -1 4202496 54932 0 0 0 5094 15 0 0 25 0 1 0 600031914 216748032 49339 33554432000 4194304 4785740 140736425602576 18446744073709551615 4416791 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25270/statm: 52917 49339 336 145 0 49962 0
[pid=25271] ppid=25268 vsize=720184 CPUtime=25.02 cores=7
/proc/25271/stat : 25271 (TNM) R 25268 25268 24128 0 -1 4202496 85171 0 0 0 2483 19 0 0 25 0 1 0 600031914 737468416 85068 33554432000 134512640 134534564 4294838384 18446744073709551615 134524469 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25271/statm: 180046 85068 91 6 0 178712 0
[pid=25272] ppid=25268 vsize=184084 CPUtime=51.09 cores=3
/proc/25272/stat : 25272 (lingeling) R 25268 25268 24128 0 -1 4202496 92547 0 0 0 5093 16 0 0 25 0 1 0 600031914 188502016 45194 33554432000 4194304 4352236 140734026021312 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25272/statm: 46021 45194 127 39 0 45101 0
[pid=25273] ppid=25268 vsize=494852 CPUtime=25.79 cores=7
/proc/25273/stat : 25273 (march_hi) R 25268 25268 24128 0 -1 4202496 488617 0 0 0 2496 83 0 0 25 0 1 0 600031914 506728448 79970 33554432000 4194304 4331988 140735137827728 18446744073709551615 4271172 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25273/statm: 123713 79970 150 34 0 122155 0
Current children cumulated CPU time (s) 204.08
Current children cumulated vsize (KiB) 2432768

[startup+102.301 s]
/proc/loadavg: 5.01 5.03 5.21 6/177 25277
/proc/meminfo: memFree=13274872/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=795536 CPUtime=102.29 cores=1

################
# More data... #
################

/proc/25273/statm: 168682 124314 150 34 0 167124 0
Current children cumulated CPU time (s) 3524.4
Current children cumulated vsize (KiB) 6982268

[startup+942.301 s]
/proc/loadavg: 5.09 5.10 5.13 6/177 25302
/proc/meminfo: memFree=11297500/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=3004472 CPUtime=942.31 cores=1
/proc/25269/stat : 25269 (cryptominisat) R 25268 25268 24128 0 -1 4202496 1010580 0 0 0 94070 161 0 0 25 0 1 0 600031914 3076579328 184453 33554432000 4194304 4587354 140733367454272 18446744073709551615 4449074 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25269/statm: 751118 184453 338 96 0 747680 0
[pid=25270] ppid=25268 vsize=2011000 CPUtime=942.3 cores=5
/proc/25270/stat : 25270 (clasp) R 25268 25268 24128 0 -1 4202496 502542 0 0 0 94078 152 0 0 25 0 1 0 600031914 2059264000 496949 33554432000 4194304 4785740 140736425602576 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25270/statm: 502750 496949 336 145 0 499795 0
[pid=25271] ppid=25268 vsize=720184 CPUtime=468.15 cores=7
/proc/25271/stat : 25271 (TNM) R 25268 25268 24128 0 -1 4202496 85171 0 0 0 46796 19 0 0 25 0 1 0 600031914 737468416 85068 33554432000 134512640 134534564 4294838384 18446744073709551615 134524508 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25271/statm: 180046 85068 91 6 0 178712 0
[pid=25272] ppid=25268 vsize=254980 CPUtime=942.31 cores=3
/proc/25272/stat : 25272 (lingeling) R 25268 25268 24128 0 -1 4202496 308906 0 0 0 94190 41 0 0 25 0 1 0 600031914 261099520 62913 33554432000 4194304 4352236 140734026021312 18446744073709551615 4207645 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25272/statm: 63745 62913 127 39 0 62825 0
[pid=25273] ppid=25268 vsize=693604 CPUtime=469.03 cores=7
/proc/25273/stat : 25273 (march_hi) R 25268 25268 24128 0 -1 4202496 537827 0 0 0 46796 107 0 0 25 0 1 0 600031914 710250496 129180 33554432000 4194304 4331988 140735137827728 18446744073709551615 4271172 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25273/statm: 173401 129180 150 34 0 171843 0
Current children cumulated CPU time (s) 3764.1
Current children cumulated vsize (KiB) 6702564

[startup+1002.3 s]
/proc/loadavg: 5.07 5.09 5.13 6/177 25304
/proc/meminfo: memFree=11160076/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=3018536 CPUtime=1002.31 cores=1
/proc/25269/stat : 25269 (cryptominisat) R 25268 25268 24128 0 -1 4202496 1083236 0 0 0 100059 172 0 0 25 0 1 0 600031914 3090980864 188518 33554432000 4194304 4587354 140733367454272 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25269/statm: 754634 188518 338 96 0 751196 0
[pid=25270] ppid=25268 vsize=2123092 CPUtime=1002.3 cores=5
/proc/25270/stat : 25270 (clasp) R 25268 25268 24128 0 -1 4202496 530420 0 0 0 100069 161 0 0 25 0 1 0 600031914 2174046208 524827 33554432000 4194304 4785740 140736425602576 18446744073709551615 4279309 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25270/statm: 530773 524827 336 145 0 527818 0
[pid=25271] ppid=25268 vsize=720184 CPUtime=498.03 cores=7
/proc/25271/stat : 25271 (TNM) R 25268 25268 24128 0 -1 4202496 85171 0 0 0 49784 19 0 0 25 0 1 0 600031914 737468416 85068 33554432000 134512640 134534564 4294838384 18446744073709551615 134524508 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25271/statm: 180046 85068 91 6 0 178712 0
[pid=25272] ppid=25268 vsize=232364 CPUtime=1002.3 cores=3
/proc/25272/stat : 25272 (lingeling) R 25268 25268 24128 0 -1 4202496 325291 0 0 0 100187 43 0 0 25 0 1 0 600031914 237940736 57259 33554432000 4194304 4352236 140734026021312 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25272/statm: 58091 57259 127 39 0 57171 0
[pid=25273] ppid=25268 vsize=723172 CPUtime=498.82 cores=7
/proc/25273/stat : 25273 (march_hi) R 25268 25268 24128 0 -1 4202496 545445 0 0 0 49772 110 0 0 25 0 1 0 600031914 740528128 136798 33554432000 4194304 4331988 140735137827728 18446744073709551615 4226645 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25273/statm: 180793 136798 150 34 0 179235 0
Current children cumulated CPU time (s) 4003.76
Current children cumulated vsize (KiB) 6835672

[startup+1062.3 s]
/proc/loadavg: 5.06 5.09 5.12 6/177 25305
/proc/meminfo: memFree=11017568/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=2621308 CPUtime=1062.31 cores=1
/proc/25269/stat : 25269 (cryptominisat) R 25268 25268 24128 0 -1 4202496 1092677 0 0 0 106057 174 0 0 25 0 1 0 600031914 2684219392 187137 33554432000 4194304 4587354 140733367454272 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25269/statm: 655327 187137 338 96 0 651889 0
[pid=25270] ppid=25268 vsize=2265252 CPUtime=1062.31 cores=5
/proc/25270/stat : 25270 (clasp) R 25268 25268 24128 0 -1 4202496 565862 0 0 0 106061 170 0 0 25 0 1 0 600031914 2319618048 560269 33554432000 4194304 4785740 140736425602576 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25270/statm: 566313 560269 336 145 0 563358 0
[pid=25271] ppid=25268 vsize=720184 CPUtime=527.84 cores=7
/proc/25271/stat : 25271 (TNM) R 25268 25268 24128 0 -1 4202496 85171 0 0 0 52765 19 0 0 25 0 1 0 600031914 737468416 85068 33554432000 134512640 134534564 4294838384 18446744073709551615 134524582 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25271/statm: 180046 85068 91 6 0 178712 0
[pid=25272] ppid=25268 vsize=232364 CPUtime=1062.31 cores=3
/proc/25272/stat : 25272 (lingeling) R 25268 25268 24128 0 -1 4202496 325291 0 0 0 106188 43 0 0 25 0 1 0 600031914 237940736 57259 33554432000 4194304 4352236 140734026021312 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25272/statm: 58091 57259 127 39 0 57171 0
[pid=25273] ppid=25268 vsize=727528 CPUtime=528.68 cores=7
/proc/25273/stat : 25273 (march_hi) R 25268 25268 24128 0 -1 4202496 546599 0 0 0 52758 110 0 0 25 0 1 0 600031914 744988672 137952 33554432000 4194304 4331988 140735137827728 18446744073709551615 4271184 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25273/statm: 181882 137952 150 34 0 180324 0
Current children cumulated CPU time (s) 4243.45
Current children cumulated vsize (KiB) 6584960

[startup+1122.3 s]
/proc/loadavg: 5.06 5.08 5.11 6/177 25307
/proc/meminfo: memFree=10864644/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=2621308 CPUtime=1122.31 cores=1
/proc/25269/stat : 25269 (cryptominisat) R 25268 25268 24128 0 -1 4202496 1096949 0 0 0 112054 177 0 0 25 0 1 0 600031914 2684219392 191361 33554432000 4194304 4587354 140733367454272 18446744073709551615 4230266 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25269/statm: 655327 191361 338 96 0 651889 0
[pid=25270] ppid=25268 vsize=2396612 CPUtime=1122.3 cores=5
/proc/25270/stat : 25270 (clasp) R 25268 25268 24128 0 -1 4202496 598626 0 0 0 112052 178 0 0 25 0 1 0 600031914 2454130688 593033 33554432000 4194304 4785740 140736425602576 18446744073709551615 4279309 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25270/statm: 599153 593033 336 145 0 596198 0
[pid=25271] ppid=25268 vsize=720184 CPUtime=557.68 cores=7
/proc/25271/stat : 25271 (TNM) R 25268 25268 24128 0 -1 4202496 85171 0 0 0 55749 19 0 0 25 0 1 0 600031914 737468416 85068 33554432000 134512640 134534564 4294838384 18446744073709551615 134524727 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25271/statm: 180046 85068 91 6 0 178712 0
[pid=25272] ppid=25268 vsize=232328 CPUtime=1122.31 cores=3
/proc/25272/stat : 25272 (lingeling) R 25268 25268 24128 0 -1 4202496 341676 0 0 0 112186 45 0 0 25 0 1 0 600031914 237903872 57250 33554432000 4194304 4352236 140734026021312 18446744073709551615 4208072 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25272/statm: 58082 57250 127 39 0 57162 0
[pid=25273] ppid=25268 vsize=733204 CPUtime=558.51 cores=7
/proc/25273/stat : 25273 (march_hi) R 25268 25268 24128 0 -1 4202496 548102 0 0 0 55740 111 0 0 25 0 1 0 600031914 750800896 139455 33554432000 4194304 4331988 140735137827728 18446744073709551615 4271172 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25273/statm: 183301 139455 150 34 0 181743 0
Current children cumulated CPU time (s) 4483.11
Current children cumulated vsize (KiB) 6721960

[startup+1182.31 s]
/proc/loadavg: 5.13 5.10 5.11 6/177 25308
/proc/meminfo: memFree=10726108/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=3054760 CPUtime=1182.32 cores=1
/proc/25269/stat : 25269 (cryptominisat) R 25268 25268 24128 0 -1 4202496 1098251 0 0 0 118053 179 0 0 25 0 1 0 600031914 3128074240 178731 33554432000 4194304 4587354 140733367454272 18446744073709551615 4238712 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25269/statm: 763690 178731 338 96 0 760252 0
[pid=25270] ppid=25268 vsize=2582388 CPUtime=1182.31 cores=5
/proc/25270/stat : 25270 (clasp) R 25268 25268 24128 0 -1 4202496 645137 0 0 0 118039 192 0 0 25 0 1 0 600031914 2644365312 639544 33554432000 4194304 4785740 140736425602576 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25270/statm: 645597 639544 336 145 0 642642 0
[pid=25271] ppid=25268 vsize=720184 CPUtime=587.53 cores=7
/proc/25271/stat : 25271 (TNM) R 25268 25268 24128 0 -1 4202496 85171 0 0 0 58734 19 0 0 25 0 1 0 600031914 737468416 85068 33554432000 134512640 134534564 4294838384 18446744073709551615 134524548 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25271/statm: 180046 85068 91 6 0 178712 0
[pid=25272] ppid=25268 vsize=232296 CPUtime=1182.32 cores=3
/proc/25272/stat : 25272 (lingeling) R 25268 25268 24128 0 -1 4202496 341676 0 0 0 118187 45 0 0 25 0 1 0 600031914 237871104 57242 33554432000 4194304 4352236 140734026021312 18446744073709551615 4278667 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25272/statm: 58074 57242 127 39 0 57154 0
[pid=25273] ppid=25268 vsize=733732 CPUtime=588.35 cores=7
/proc/25273/stat : 25273 (march_hi) R 25268 25268 24128 0 -1 4202496 548262 0 0 0 58723 112 0 0 25 0 1 0 600031914 751341568 139615 33554432000 4194304 4331988 140735137827728 18446744073709551615 4226647 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25273/statm: 183433 139615 150 34 0 181875 0
Current children cumulated CPU time (s) 4722.83
Current children cumulated vsize (KiB) 7341684



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 5.16 5.10 5.11 6/177 25309
/proc/meminfo: memFree=10653060/32951124 swapFree=67102140/67111528
[pid=25268] ppid=25266 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25268/stat : 25268 (ppfolio) S 25266 25268 24128 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 600031914 18763776 389 33554432000 4194304 4287625 140736725998816 18446744073709551615 216735198735 0 0 4096 65536 4365149952129630207 0 0 17 5 0 0 0
/proc/25268/statm: 4581 389 341 23 0 91 0
[pid=25269] ppid=25268 vsize=2611332 CPUtime=1201.72 cores=1
/proc/25269/stat : 25269 (cryptominisat) R 25268 25268 24128 0 -1 4202496 1107056 0 0 0 119992 180 0 0 25 0 1 0 600031914 2674003968 187518 33554432000 4194304 4587354 140733367454272 18446744073709551615 4230266 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25269/statm: 652833 187518 338 96 0 649395 0
[pid=25270] ppid=25268 vsize=2620832 CPUtime=1201.71 cores=5
/proc/25270/stat : 25270 (clasp) R 25268 25268 24128 0 -1 4202496 654729 0 0 0 119975 196 0 0 25 0 1 0 600031914 2683731968 649136 33554432000 4194304 4785740 140736425602576 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25270/statm: 655208 649136 336 145 0 652253 0
[pid=25271] ppid=25268 vsize=720184 CPUtime=597.16 cores=7
/proc/25271/stat : 25271 (TNM) R 25268 25268 24128 0 -1 4202496 85171 0 0 0 59697 19 0 0 25 0 1 0 600031914 737468416 85068 33554432000 134512640 134534564 4294838384 18446744073709551615 134524500 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25271/statm: 180046 85068 91 6 0 178712 0
[pid=25272] ppid=25268 vsize=232296 CPUtime=1201.72 cores=3
/proc/25272/stat : 25272 (lingeling) R 25268 25268 24128 0 -1 4202496 358061 0 0 0 120125 47 0 0 25 0 1 0 600031914 237871104 57242 33554432000 4194304 4352236 140734026021312 18446744073709551615 4207627 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25272/statm: 58074 57242 127 39 0 57154 0
[pid=25273] ppid=25268 vsize=733732 CPUtime=598.01 cores=7
/proc/25273/stat : 25273 (march_hi) R 25268 25268 24128 0 -1 4202496 548271 0 0 0 59689 112 0 0 25 0 1 0 600031914 751341568 139624 33554432000 4194304 4331988 140735137827728 18446744073709551615 4271172 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25273/statm: 183433 139624 150 34 0 181875 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 6936700

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25268 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=5999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=9998
# CPU time returned by wait4() is 0.015997
# while last known CPU time is 4800.32
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.32
CPU user time (s): 4794.78
CPU system time (s): 5.54
CPU usage (%): 399.456
Max. virtual memory (cumulated for all children) (KiB): 9332380

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.005999
system time used= 0.009998
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 520
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 202
involuntary context switches= 7

runsolver used 1.76673 second user time and 4.78827 second system time

The end

Launcher Data 

Begin job on node119 at 2011-04-29 19:26:53
IDJOB=3293676
IDBENCH=88165
IDSOLVER=1612
FILE ID=node119/3293676-1304098012
RUNJOBID= node119-1304090423-24478
PBS_JOBID= 13170550
Free space on /tmp= 71724 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT-Race-2010-CNF/hardware-verification/velev/2dlx_ca_bp_f_liveness.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3293676-1304098012/watcher-3293676-1304098012 -o /tmp/evaluation-result-3293676-1304098012/solver-3293676-1304098012 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3293676-1304098012.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 99cd8de7f06ce8c087a5c7fb299dfb62
RANDOM SEED=711314683

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15132812 kB
Buffers:       2210584 kB
Cached:       14961124 kB
SwapCached:       5504 kB
Active:        4769980 kB
Inactive:     12462976 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15132812 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:          102832 kB
Writeback:           0 kB
AnonPages:       58352 kB
Mapped:          14564 kB
Slab:           523180 kB
PageTables:       4456 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   188740 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71716 MiB
End job on node119 at 2011-04-29 19:46:56