Trace number 3291606

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.21 1201.71

General information on the benchmark

NameSAT03/industrial/schuppan/l2s/
valves-gates-1-k617-unsat.shuffled-as.sat03-412.cnf
MD5SUMcb369c4583eb075a3a8cb5a04701e120
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark4716.27
Satisfiable
(Un)Satisfiability was proved
Number of variables985042
Number of clauses3113540
Sum of the clauses size7976612
Maximum clause length618
Minimum clause length1
Number of clauses of size 1158
Number of clauses of size 21490897
Number of clauses of size 31578048
Number of clauses of size 428388
Number of clauses of size 59876
Number of clauses of size over 56173

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 4
0.00/0.00	Starting TNM 2009 on core(s) 6
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 6

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3291606-1304093052/watcher-3291606-1304093052 -o /tmp/evaluation-result-3291606-1304093052/solver-3291606-1304093052 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3291606-1304093052.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.85 8.25 8.64 6/181 14976
/proc/meminfo: memFree=22379472/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) R 14974 14976 14150 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0

[startup+0.059104 s]
/proc/loadavg: 7.85 8.25 8.64 6/181 14976
/proc/meminfo: memFree=22379472/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) R 14974 14976 14150 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.101098 s]
/proc/loadavg: 7.85 8.25 8.64 6/181 14976
/proc/meminfo: memFree=22379472/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) R 14974 14976 14150 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.301062 s]
/proc/loadavg: 7.85 8.25 8.64 6/181 14976
/proc/meminfo: memFree=22379472/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700987 s]
/proc/loadavg: 7.85 8.25 8.64 6/181 14976
/proc/meminfo: memFree=22379472/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50084 s]
/proc/loadavg: 8.02 8.28 8.65 10/187 14982
/proc/meminfo: memFree=21681804/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=323612 CPUtime=1.48 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 69171 0 0 0 137 11 0 0 25 0 1 0 520906001 331378688 67182 33554432000 4194304 4587354 140737413219744 18446744073709551615 4414927 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 80903 67183 289 96 0 77465 0
[pid=14978] ppid=14976 vsize=349728 CPUtime=1.49 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 106362 0 0 0 135 14 0 0 25 0 1 0 520906001 358121472 81489 33554432000 4194304 4785740 140735712155872 18446744073709551615 4489193 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 87432 81489 312 145 0 84477 0
[pid=14979] ppid=14976 vsize=469028 CPUtime=0.69 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 7087 0 0 0 68 1 0 0 25 0 1 0 520906001 480284672 6985 33554432000 134512640 134534564 4287600688 18446744073709551615 9910445 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 117257 6985 90 6 0 115923 0
[pid=14980] ppid=14976 vsize=99712 CPUtime=1.48 cores=2
/proc/14980/stat : 14980 (lingeling) R 14976 14976 14150 0 -1 4202496 24278 0 0 0 145 3 0 0 25 0 1 0 520906001 102105088 24096 33554432000 4194304 4352236 140736974729120 18446744073709551615 244446945024 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14980/statm: 24928 24096 101 39 0 24008 0
[pid=14981] ppid=14976 vsize=178348 CPUtime=0.78 cores=6
/proc/14981/stat : 14981 (march_hi) R 14976 14976 14150 0 -1 4194304 40315 0 0 0 72 6 0 0 25 0 1 0 520906001 182628352 40210 33554432000 4194304 4331988 140735053347248 18446744073709551615 244446494049 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14981/statm: 44587 40210 113 34 0 43029 0
Current children cumulated CPU time (s) 5.92
Current children cumulated vsize (KiB) 1438752

[startup+3.10053 s]
/proc/loadavg: 8.02 8.28 8.65 10/187 14982
/proc/meminfo: memFree=21354192/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 18 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=387812 CPUtime=3.09 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 89574 0 0 0 293 16 0 0 25 0 1 0 520906001 397119488 86918 33554432000 4194304 4587354 140737413219744 18446744073709551615 4230345 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 96953 86918 306 96 0 93515 0
[pid=14978] ppid=14976 vsize=415240 CPUtime=3.09 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 122525 0 0 0 293 16 0 0 25 0 1 0 520906001 425205760 97652 33554432000 4194304 4785740 140735712155872 18446744073709551615 4489193 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 103810 97652 313 145 0 100855 0
[pid=14979] ppid=14976 vsize=488696 CPUtime=1.49 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 14869 0 0 0 146 3 0 0 25 0 1 0 520906001 500424704 14767 33554432000 134512640 134534564 4287600688 18446744073709551615 9910620 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 122174 14767 90 6 0 120840 0
[pid=14980] ppid=14976 vsize=134016 CPUtime=3.09 cores=2
/proc/14980/stat : 14980 (lingeling) R 14976 14976 14150 0 -1 4202496 32854 0 0 0 303 6 0 0 25 0 1 0 520906001 137232384 32672 33554432000 4194304 4352236 140736974729120 18446744073709551615 244446945024 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14980/statm: 33504 32672 101 39 0 32584 0
[pid=14981] ppid=14976 vsize=219532 CPUtime=1.57 cores=6
/proc/14981/stat : 14981 (march_hi) R 14976 14976 14150 0 -1 4194304 51883 0 0 0 148 9 0 0 25 0 1 0 520906001 224800768 51778 33554432000 4194304 4331988 140735053347248 18446744073709551615 244446478901 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14981/statm: 54883 51778 113 34 0 53325 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 1663620

[startup+6.30094 s]
/proc/loadavg: 8.10 8.29 8.65 10/187 14982
/proc/meminfo: memFree=20875792/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 18 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=387812 CPUtime=6.29 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 89622 0 0 0 613 16 0 0 25 0 1 0 520906001 397119488 86966 33554432000 4194304 4587354 140737413219744 18446744073709551615 4238271 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 96953 86966 306 96 0 93515 0
[pid=14978] ppid=14976 vsize=444652 CPUtime=6.29 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 129584 0 0 0 612 17 0 0 25 0 1 0 520906001 455323648 104711 33554432000 4194304 4785740 140735712155872 18446744073709551615 4495638 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 111163 104711 318 145 0 108208 0
[pid=14979] ppid=14976 vsize=650260 CPUtime=3.09 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 66799 0 0 0 298 11 0 0 25 0 1 0 520906001 665866240 66696 33554432000 134512640 134534564 4287600688 18446744073709551615 134520097 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 162565 66696 91 6 0 161231 0
[pid=14980] ppid=14976 vsize=277068 CPUtime=6.29 cores=2
/proc/14980/stat : 14980 (lingeling) R 14976 14976 14150 0 -1 4202496 70580 0 0 0 618 11 0 0 25 0 1 0 520906001 283717632 68445 33554432000 4194304 4352236 140736974729120 18446744073709551615 4207908 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14980/statm: 69267 68445 111 39 0 68347 0
[pid=14981] ppid=14976 vsize=565180 CPUtime=3.16 cores=6
/proc/14981/stat : 14981 (march_hi) R 14976 14976 14150 0 -1 4194304 108577 0 0 0 300 16 0 0 25 0 1 0 520906001 578744320 108470 33554432000 4194304 4331988 140735053347248 18446744073709551615 4251678 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14981/statm: 141295 108470 117 34 0 139737 0
Current children cumulated CPU time (s) 25.12
Current children cumulated vsize (KiB) 2343296

[startup+12.7007 s]
/proc/loadavg: 8.17 8.30 8.65 10/187 14982
/proc/meminfo: memFree=20501944/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=612772 CPUtime=12.69 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 145114 0 0 0 1244 25 0 0 25 0 1 0 520906001 627478528 142458 33554432000 4194304 4587354 140737413219744 18446744073709551615 4421100 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 153193 142458 311 96 0 149755 0
[pid=14978] ppid=14976 vsize=510912 CPUtime=12.68 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 145901 0 0 0 1248 20 0 0 25 0 1 0 520906001 523173888 121028 33554432000 4194304 4785740 140735712155872 18446744073709551615 4484287 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 127728 121028 319 145 0 124773 0
[pid=14979] ppid=14976 vsize=650260 CPUtime=6.29 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 66800 0 0 0 618 11 0 0 25 0 1 0 520906001 665866240 66697 33554432000 134512640 134534564 4287600688 18446744073709551615 134524754 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 162565 66697 91 6 0 161231 0
[pid=14980] ppid=14976 vsize=235796 CPUtime=12.69 cores=2
/proc/14980/stat : 14980 (lingeling) R 14976 14976 14150 0 -1 4202496 120961 0 0 0 1252 17 0 0 25 0 1 0 520906001 241455104 58101 33554432000 4194304 4352236 140736974729120 18446744073709551615 4256354 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14980/statm: 58949 58101 117 39 0 58029 0
[pid=14981] ppid=14976 vsize=466976 CPUtime=6.32 cores=6
/proc/14981/stat : 14981 (march_hi) R 14976 14976 14150 0 -1 4194304 117615 0 0 0 615 17 0 0 25 0 1 0 520906001 478183424 84865 33554432000 4194304 4331988 140735053347248 18446744073709551615 4262122 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14981/statm: 116744 84865 126 34 0 115186 0
Current children cumulated CPU time (s) 50.67
Current children cumulated vsize (KiB) 2495040

[startup+25.5004 s]
/proc/loadavg: 8.43 8.35 8.66 10/187 14983
/proc/meminfo: memFree=20196576/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=612772 CPUtime=25.49 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 145114 0 0 0 2524 25 0 0 25 0 1 0 520906001 627478528 142458 33554432000 4194304 4587354 140737413219744 18446744073709551615 4421069 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 153193 142458 311 96 0 149755 0
[pid=14978] ppid=14976 vsize=551640 CPUtime=25.48 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 172778 0 0 0 2522 26 0 0 25 0 1 0 520906001 564879360 132205 33554432000 4194304 4785740 140735712155872 18446744073709551615 4319144 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 137910 132205 332 145 0 134955 0
[pid=14979] ppid=14976 vsize=650260 CPUtime=12.62 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 66800 0 0 0 1251 11 0 0 25 0 1 0 520906001 665866240 66697 33554432000 134512640 134534564 4287600688 18446744073709551615 134524508 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 162565 66697 91 6 0 161231 0
[pid=14980] ppid=14976 vsize=276900 CPUtime=25.48 cores=2
/proc/14980/stat : 14980 (lingeling) R 14976 14976 14150 0 -1 4202496 131251 0 0 0 2530 18 0 0 25 0 1 0 520906001 283545600 68391 33554432000 4194304 4352236 140736974729120 18446744073709551615 4215200 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14980/statm: 69225 68391 125 39 0 68305 0
[pid=14981] ppid=14976 vsize=666396 CPUtime=12.71 cores=6
/proc/14981/stat : 14981 (march_hi) R 14976 14976 14150 0 -1 4194304 167528 0 0 0 1247 24 0 0 25 0 1 0 520906001 682389504 134769 33554432000 4194304 4331988 140735053347248 18446744073709551615 244446594252 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14981/statm: 166599 134769 129 34 0 165041 0
Current children cumulated CPU time (s) 101.78
Current children cumulated vsize (KiB) 2776292

[startup+51.1006 s]
/proc/loadavg: 8.68 8.42 8.67 10/187 14983
/proc/meminfo: memFree=19869504/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=985256 CPUtime=51.08 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 292389 0 0 0 5056 52 0 0 25 0 1 0 520906001 1008902144 195063 33554432000 4194304 4587354 140737413219744 18446744073709551615 4409519 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 246314 195063 328 96 0 242876 0
[pid=14978] ppid=14976 vsize=551640 CPUtime=51.08 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 172816 0 0 0 5082 26 0 0 25 0 1 0 520906001 564879360 132243 33554432000 4194304 4785740 140735712155872 18446744073709551615 4552646 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 137910 132243 340 145 0 134955 0
[pid=14979] ppid=14976 vsize=701740 CPUtime=25.39 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 117246 0 0 0 2520 19 0 0 25 0 1 0 520906001 718581760 117143 33554432000 134512640 134534564 4287600688 18446744073709551615 134529964 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 175435 117143 95 6 0 174101 0
[pid=14980] ppid=14976 vsize=179448 CPUtime=51.08 cores=2

################
# More data... #
################

/proc/14981/statm: 523691 276756 1135 34 0 522133 0
Current children cumulated CPU time (s) 3764
Current children cumulated vsize (KiB) 6837916

[startup+1002.3 s]
/proc/loadavg: 10.18 9.67 9.19 11/188 15042
/proc/meminfo: memFree=19377460/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=2934396 CPUtime=1002.29 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 413852 0 0 0 100141 88 0 0 25 0 1 0 520906001 3004821504 216659 33554432000 4194304 4587354 140737413219744 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 733599 216659 337 96 0 730161 0
[pid=14978] ppid=14976 vsize=935028 CPUtime=1002.21 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 268212 0 0 0 100150 71 0 0 25 0 1 0 520906001 957468672 227639 33554432000 4194304 4785740 140735712155872 18446744073709551615 4309555 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 233757 227639 340 145 0 230802 0
[pid=14979] ppid=14976 vsize=705576 CPUtime=498.41 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 117248 0 0 0 49822 19 0 0 25 0 1 0 520906001 722509824 117145 33554432000 134512640 134534564 4287600688 18446744073709551615 134530593 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 176394 117145 95 6 0 175060 0
[pid=14980] ppid=14976 vsize=186700 CPUtime=1002.3 cores=2
/proc/14980/stat : 14980 (lingeling) R 14976 14976 14150 0 -1 4202496 156620 0 0 0 100206 24 0 0 25 0 1 0 520906001 191180800 45843 33554432000 4194304 4352236 140736974729120 18446744073709551615 4328400 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14980/statm: 46675 45843 127 39 0 45755 0
[pid=14981] ppid=14976 vsize=2098916 CPUtime=498.44 cores=6
/proc/14981/stat : 14981 (march_hi) R 14976 14976 14150 0 -1 4202496 538729 0 0 0 49769 75 0 0 25 0 1 0 520906001 2149289984 277707 33554432000 4194304 4331988 140735053347248 18446744073709551615 4237136 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14981/statm: 524729 277707 1135 34 0 523171 0
Current children cumulated CPU time (s) 4003.65
Current children cumulated vsize (KiB) 6878940

[startup+1062.3 s]
/proc/loadavg: 10.17 9.76 9.25 11/188 15043
/proc/meminfo: memFree=19355724/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=951296 CPUtime=1062.3 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 438111 0 0 0 106136 94 0 0 25 0 1 0 520906001 974127104 208595 33554432000 4194304 4587354 140737413219744 18446744073709551615 4238755 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 237824 208595 337 96 0 234386 0
[pid=14978] ppid=14976 vsize=987924 CPUtime=1062.22 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 281447 0 0 0 106144 78 0 0 25 0 1 0 520906001 1011634176 240874 33554432000 4194304 4785740 140735712155872 18446744073709551615 4553068 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 246981 240874 340 145 0 244026 0
[pid=14979] ppid=14976 vsize=705576 CPUtime=528.21 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 117248 0 0 0 52802 19 0 0 25 0 1 0 520906001 722509824 117145 33554432000 134512640 134534564 4287600688 18446744073709551615 134530151 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 176394 117145 95 6 0 175060 0
[pid=14980] ppid=14976 vsize=186696 CPUtime=1062.3 cores=2
/proc/14980/stat : 14980 (lingeling) R 14976 14976 14150 0 -1 4202496 156620 0 0 0 106206 24 0 0 25 0 1 0 520906001 191176704 45842 33554432000 4194304 4352236 140736974729120 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14980/statm: 46674 45842 127 39 0 45754 0
[pid=14981] ppid=14976 vsize=2100964 CPUtime=528.31 cores=6
/proc/14981/stat : 14981 (march_hi) R 14976 14976 14150 0 -1 4202496 539229 0 0 0 52756 75 0 0 25 0 1 0 520906001 2151387136 278207 33554432000 4194304 4331988 140735053347248 18446744073709551615 4243165 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14981/statm: 525241 278207 1135 34 0 523683 0
Current children cumulated CPU time (s) 4243.34
Current children cumulated vsize (KiB) 4950780

[startup+1122.3 s]
/proc/loadavg: 10.18 9.83 9.30 11/188 15045
/proc/meminfo: memFree=19258480/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=1213444 CPUtime=1122.31 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 445631 0 0 0 112132 99 0 0 25 0 1 0 520906001 1242566656 216095 33554432000 4194304 4587354 140737413219744 18446744073709551615 4239225 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 303361 216095 337 96 0 299923 0
[pid=14978] ppid=14976 vsize=1038152 CPUtime=1122.21 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 293895 0 0 0 112139 82 0 0 25 0 1 0 520906001 1063067648 253322 33554432000 4194304 4785740 140735712155872 18446744073709551615 4552646 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 259538 253322 340 145 0 256583 0
[pid=14979] ppid=14976 vsize=705576 CPUtime=558.07 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 117248 0 0 0 55788 19 0 0 25 0 1 0 520906001 722509824 117145 33554432000 134512640 134534564 4287600688 18446744073709551615 134528600 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 176394 117145 95 6 0 175060 0
[pid=14980] ppid=14976 vsize=192624 CPUtime=1122.3 cores=2
/proc/14980/stat : 14980 (lingeling) R 14976 14976 14150 0 -1 4202496 158110 0 0 0 112206 24 0 0 25 0 1 0 520906001 197246976 47324 33554432000 4194304 4352236 140736974729120 18446744073709551615 4208899 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14980/statm: 48156 47324 127 39 0 47236 0
[pid=14981] ppid=14976 vsize=2101132 CPUtime=558.13 cores=6
/proc/14981/stat : 14981 (march_hi) R 14976 14976 14150 0 -1 4202496 539249 0 0 0 55738 75 0 0 25 0 1 0 520906001 2151559168 278227 33554432000 4194304 4331988 140735053347248 18446744073709551615 4243090 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14981/statm: 525283 278227 1135 34 0 523725 0
Current children cumulated CPU time (s) 4483.02
Current children cumulated vsize (KiB) 5269252

[startup+1182.3 s]
/proc/loadavg: 10.20 9.89 9.35 11/188 15047
/proc/meminfo: memFree=19266260/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=3148968 CPUtime=1182.31 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 466112 0 0 0 118128 103 0 0 25 0 1 0 520906001 3224543232 209525 33554432000 4194304 4587354 140737413219744 18446744073709551615 4248764 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 787242 209525 337 96 0 783804 0
[pid=14978] ppid=14976 vsize=1076516 CPUtime=1182.2 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 303411 0 0 0 118134 86 0 0 25 0 1 0 520906001 1102352384 262838 33554432000 4194304 4785740 140735712155872 18446744073709551615 4416832 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 269129 262838 340 145 0 266174 0
[pid=14979] ppid=14976 vsize=705576 CPUtime=587.91 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 117248 0 0 0 58772 19 0 0 25 0 1 0 520906001 722509824 117145 33554432000 134512640 134534564 4287600688 18446744073709551615 134530309 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 176394 117145 95 6 0 175060 0
[pid=14980] ppid=14976 vsize=192624 CPUtime=1182.3 cores=2
/proc/14980/stat : 14980 (lingeling) R 14976 14976 14150 0 -1 4202496 158110 0 0 0 118206 24 0 0 25 0 1 0 520906001 197246976 47324 33554432000 4194304 4352236 140736974729120 18446744073709551615 4278623 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14980/statm: 48156 47324 127 39 0 47236 0
[pid=14981] ppid=14976 vsize=2101132 CPUtime=587.97 cores=6
/proc/14981/stat : 14981 (march_hi) R 14976 14976 14150 0 -1 4202496 539249 0 0 0 58722 75 0 0 25 0 1 0 520906001 2151559168 278227 33554432000 4194304 4331988 140735053347248 18446744073709551615 4242538 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14981/statm: 525283 278227 1135 34 0 523725 0
Current children cumulated CPU time (s) 4722.69
Current children cumulated vsize (KiB) 7243140



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 10.21 9.91 9.37 11/188 15047
/proc/meminfo: memFree=19224952/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=3148968 CPUtime=1201.71 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 467219 0 0 0 120068 103 0 0 25 0 1 0 520906001 3224543232 210622 33554432000 4194304 4587354 140737413219744 18446744073709551615 4230209 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 787242 210622 337 96 0 783804 0
[pid=14978] ppid=14976 vsize=1089644 CPUtime=1201.61 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 306650 0 0 0 120074 87 0 0 25 0 1 0 520906001 1115795456 266077 33554432000 4194304 4785740 140735712155872 18446744073709551615 4552910 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 272411 266077 340 145 0 269456 0
[pid=14979] ppid=14976 vsize=705576 CPUtime=597.55 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 117248 0 0 0 59736 19 0 0 25 0 1 0 520906001 722509824 117145 33554432000 134512640 134534564 4287600688 18446744073709551615 134530151 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 176394 117145 95 6 0 175060 0
[pid=14980] ppid=14976 vsize=192624 CPUtime=1201.71 cores=2
/proc/14980/stat : 14980 (lingeling) R 14976 14976 14150 0 -1 4202496 158110 0 0 0 120147 24 0 0 25 0 1 0 520906001 197246976 47324 33554432000 4194304 4352236 140736974729120 18446744073709551615 4329024 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14980/statm: 48156 47324 127 39 0 47236 0
[pid=14981] ppid=14976 vsize=2107520 CPUtime=597.63 cores=6
/proc/14981/stat : 14981 (march_hi) R 14976 14976 14150 0 -1 4202496 544839 0 0 0 59687 76 0 0 25 0 1 0 520906001 2158100480 282856 33554432000 4194304 4331988 140735053347248 18446744073709551615 4237393 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14981/statm: 526880 282856 177 34 0 525322 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 7262656

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 10.21 9.91 9.37 11/188 15047
/proc/meminfo: memFree=19224952/32951124 swapFree=67111524/67111528
[pid=14976] ppid=14974 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14976/stat : 14976 (ppfolio) S 14974 14976 14150 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520906001 18763776 390 33554432000 4194304 4287625 140736314726288 18446744073709551615 244446965263 0 0 4096 65536 4579975695060434943 0 0 17 4 0 0 0
/proc/14976/statm: 4581 390 341 23 0 91 0
[pid=14977] ppid=14976 vsize=3148968 CPUtime=1201.71 cores=0
/proc/14977/stat : 14977 (cryptominisat) R 14976 14976 14150 0 -1 4202496 467219 0 0 0 120068 103 0 0 25 0 1 0 520906001 3224543232 210622 33554432000 4194304 4587354 140737413219744 18446744073709551615 4230209 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14977/statm: 787242 210622 337 96 0 783804 0
[pid=14978] ppid=14976 vsize=1089644 CPUtime=1201.61 cores=4
/proc/14978/stat : 14978 (clasp) R 14976 14976 14150 0 -1 4202496 306650 0 0 0 120074 87 0 0 25 0 1 0 520906001 1115795456 266077 33554432000 4194304 4785740 140735712155872 18446744073709551615 4552910 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14978/statm: 272411 266077 340 145 0 269456 0
[pid=14979] ppid=14976 vsize=705576 CPUtime=597.55 cores=6
/proc/14979/stat : 14979 (TNM) R 14976 14976 14150 0 -1 4202496 117248 0 0 0 59736 19 0 0 25 0 1 0 520906001 722509824 117145 33554432000 134512640 134534564 4287600688 18446744073709551615 134530151 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14979/statm: 176394 117145 95 6 0 175060 0
[pid=14980] ppid=14976 vsize=192624 CPUtime=1201.71 cores=2
/proc/14980/stat : 14980 (lingeling) R 14976 14976 14150 0 -1 4202496 158110 0 0 0 120147 24 0 0 25 0 1 0 520906001 197246976 47324 33554432000 4194304 4352236 140736974729120 18446744073709551615 4329024 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14980/statm: 48156 47324 127 39 0 47236 0
[pid=14981] ppid=14976 vsize=2107520 CPUtime=597.63 cores=6
/proc/14981/stat : 14981 (march_hi) R 14976 14976 14150 0 -1 4202496 544839 0 0 0 59687 76 0 0 25 0 1 0 520906001 2158100480 282856 33554432000 4194304 4331988 140735053347248 18446744073709551615 4237393 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14981/statm: 526880 282856 177 34 0 525322 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 7262656

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 14976 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=1999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=999
# CPU time returned by wait4() is 0.002998
# while last known CPU time is 4800.21
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.21
CPU user time (s): 4797.12
CPU system time (s): 3.09
CPU usage (%): 399.447
Max. virtual memory (cumulated for all children) (KiB): 7575880

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.001999
system time used= 0.000999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 523
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 157
involuntary context switches= 7

runsolver used 1.9947 second user time and 4.57031 second system time

The end

Launcher Data 

Begin job on node105 at 2011-04-29 18:04:16
IDJOB=3291606
IDBENCH=84238
IDSOLVER=1612
FILE ID=node105/3291606-1304093052
RUNJOBID= node105-1304087809-14168
PBS_JOBID= 13170561
Free space on /tmp= 71868 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT03/industrial/schuppan/l2s/valves-gates-1-k617-unsat.shuffled-as.sat03-412.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3291606-1304093052/watcher-3291606-1304093052 -o /tmp/evaluation-result-3291606-1304093052/solver-3291606-1304093052 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3291606-1304093052.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cb369c4583eb075a3a8cb5a04701e120
RANDOM SEED=1002397293

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      22379752 kB
Buffers:       2156348 kB
Cached:        7170996 kB
SwapCached:          4 kB
Active:        1238636 kB
Inactive:      8782656 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      22379752 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           75248 kB
Writeback:           0 kB
AnonPages:      693720 kB
Mapped:          16476 kB
Slab:           484500 kB
PageTables:       6392 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  1442380 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71856 MiB
End job on node105 at 2011-04-29 18:24:19