Trace number 3251567

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.29 1201.92

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m52-q1041.25-n50-p15-s99099953.cnf
MD5SUMdda220d08783aabf4e7a9e99eeeccae6
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2444
Number of clauses498568
Sum of the clauses size999480
Maximum clause length52
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2498518
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 550

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.00	Starting clasp 1.3.6 on core(s) 5
0.00/0.00	Starting TNM 2009 on core(s) 7
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.00	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3251567-1304100226/watcher-3251567-1304100226 -o /tmp/evaluation-result-3251567-1304100226/solver-3251567-1304100226 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3251567-1304100226.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.20 10.17 9.74 6/182 2586
/proc/meminfo: memFree=29881360/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) R 2584 2586 2052 0 -1 4194304 506 0 0 0 0 0 0 0 25 0 1 0 521623354 18763776 387 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921597333 0 0 4096 65536 0 0 0 17 5 0 0 0
/proc/2586/statm: 4581 387 339 23 0 91 0

[startup+0.0868 s]
/proc/loadavg: 10.20 10.17 9.74 6/182 2586
/proc/meminfo: memFree=29881360/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100798 s]
/proc/loadavg: 10.20 10.17 9.74 6/182 2586
/proc/meminfo: memFree=29881360/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300762 s]
/proc/loadavg: 10.20 10.17 9.74 6/182 2586
/proc/meminfo: memFree=29881360/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 23 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700693 s]
/proc/loadavg: 10.20 10.17 9.74 6/182 2586
/proc/meminfo: memFree=29881360/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 23 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50054 s]
/proc/loadavg: 10.27 10.18 9.74 11/188 2594
/proc/meminfo: memFree=29698272/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 23 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=97988 CPUtime=1.49 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 20965 0 0 0 147 2 0 0 25 0 1 0 521623354 100339712 17905 33554432000 4194304 4587354 140734210868192 18446744073709551615 222921473684 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 24497 17905 315 96 0 21059 0
[pid=2588] ppid=2586 vsize=45856 CPUtime=1.48 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 10980 0 0 0 147 1 0 0 25 0 1 0 521623354 46956544 8179 33554432000 4194304 4785740 140734861304096 18446744073709551615 4495561 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 11464 8179 313 145 0 8509 0
[pid=2589] ppid=2586 vsize=475228 CPUtime=0.7 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 8049 0 0 0 70 0 0 0 25 0 1 0 521623354 486633472 7946 33554432000 134512640 134534564 4289980672 18446744073709551615 134524582 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 118807 7946 91 6 0 117473 0
[pid=2590] ppid=2586 vsize=25536 CPUtime=1.48 cores=3
/proc/2590/stat : 2590 (lingeling) R 2586 2586 2052 0 -1 4202496 7727 0 0 0 148 0 0 0 25 0 1 0 521623354 26148864 5542 33554432000 4194304 4352236 140733525019408 18446744073709551615 4267938 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2590/statm: 6384 5542 123 39 0 5464 0
[pid=2591] ppid=2586 vsize=55700 CPUtime=0.77 cores=7
/proc/2591/stat : 2591 (march_hi) R 2586 2586 2052 0 -1 4202496 26167 0 0 0 73 4 0 0 25 0 1 0 521623354 57036800 7637 33554432000 4194304 4331988 140735785846976 18446744073709551615 4257049 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2591/statm: 13925 7637 149 34 0 12367 0
Current children cumulated CPU time (s) 5.92
Current children cumulated vsize (KiB) 718632

[startup+3.10125 s]
/proc/loadavg: 10.27 10.18 9.74 11/188 2594
/proc/meminfo: memFree=29691140/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=100128 CPUtime=3.09 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 26886 0 0 0 306 3 0 0 25 0 1 0 521623354 102531072 19945 33554432000 4194304 4587354 140734210868192 18446744073709551615 222921473688 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 25032 19945 326 96 0 21594 0
[pid=2588] ppid=2586 vsize=46120 CPUtime=3.09 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 11070 0 0 0 308 1 0 0 25 0 1 0 521623354 47226880 8269 33554432000 4194304 4785740 140734861304096 18446744073709551615 4482817 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 11530 8269 316 145 0 8575 0
[pid=2589] ppid=2586 vsize=475228 CPUtime=1.5 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 8049 0 0 0 150 0 0 0 25 0 1 0 521623354 486633472 7946 33554432000 134512640 134534564 4289980672 18446744073709551615 134524483 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 118807 7946 91 6 0 117473 0
[pid=2590] ppid=2586 vsize=26688 CPUtime=3.09 cores=3
/proc/2590/stat : 2590 (lingeling) R 2586 2586 2052 0 -1 4202496 8018 0 0 0 309 0 0 0 25 0 1 0 521623354 27328512 5833 33554432000 4194304 4352236 140733525019408 18446744073709551615 4328832 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2590/statm: 6672 5833 126 39 0 5752 0
[pid=2591] ppid=2586 vsize=72224 CPUtime=1.56 cores=7
/proc/2591/stat : 2591 (march_hi) R 2586 2586 2052 0 -1 4202496 38897 0 0 0 151 5 0 0 25 0 1 0 521623354 73957376 8377 33554432000 4194304 4331988 140735785846976 18446744073709551615 4226263 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2591/statm: 18056 8377 170 34 0 16498 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 738712

[startup+6.30067 s]
/proc/loadavg: 10.27 10.18 9.74 11/188 2594
/proc/meminfo: memFree=29675168/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=282760 CPUtime=6.29 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 30797 0 0 0 625 4 0 0 25 0 1 0 521623354 289546240 20412 33554432000 4194304 4587354 140734210868192 18446744073709551615 4238668 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 70690 20412 331 96 0 67252 0
[pid=2588] ppid=2586 vsize=42364 CPUtime=6.29 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 11708 0 0 0 628 1 0 0 25 0 1 0 521623354 43380736 7933 33554432000 4194304 4785740 140734861304096 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 10591 7933 340 145 0 7636 0
[pid=2589] ppid=2586 vsize=475228 CPUtime=3.1 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 8049 0 0 0 310 0 0 0 25 0 1 0 521623354 486633472 7946 33554432000 134512640 134534564 4289980672 18446744073709551615 134524508 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 118807 7946 91 6 0 117473 0
[pid=2590] ppid=2586 vsize=41536 CPUtime=6.28 cores=3
/proc/2590/stat : 2590 (lingeling) R 2586 2586 2052 0 -1 4202496 10707 0 0 0 628 0 0 0 25 0 1 0 521623354 42532864 8522 33554432000 4194304 4352236 140733525019408 18446744073709551615 4267912 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2590/statm: 10384 8522 126 39 0 9464 0
[pid=2591] ppid=2586 vsize=72356 CPUtime=3.14 cores=7
/proc/2591/stat : 2591 (march_hi) R 2586 2586 2052 0 -1 4202496 38910 0 0 0 309 5 0 0 25 0 1 0 521623354 74092544 8390 33554432000 4194304 4331988 140735785846976 18446744073709551615 4228226 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2591/statm: 18089 8390 171 34 0 16531 0
Current children cumulated CPU time (s) 25.1
Current children cumulated vsize (KiB) 932568

[startup+12.7005 s]
/proc/loadavg: 10.31 10.19 9.75 13/189 2595
/proc/meminfo: memFree=29596676/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=282760 CPUtime=12.54 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 30817 0 0 0 1250 4 0 0 25 0 1 0 521623354 289546240 20432 33554432000 4194304 4587354 140734210868192 18446744073709551615 4365356 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 70690 20432 337 96 0 67252 0
[pid=2588] ppid=2586 vsize=81068 CPUtime=12.68 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 21276 0 0 0 1265 3 0 0 25 0 1 0 521623354 83013632 17501 33554432000 4194304 4785740 140734861304096 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 20267 17501 340 145 0 17312 0
[pid=2589] ppid=2586 vsize=479188 CPUtime=6.3 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 11667 0 0 0 630 0 0 0 25 0 1 0 521623354 490688512 11564 33554432000 134512640 134534564 4289980672 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 119797 11564 95 6 0 118463 0
[pid=2590] ppid=2586 vsize=48000 CPUtime=12.69 cores=3
/proc/2590/stat : 2590 (lingeling) R 2586 2586 2052 0 -1 4202496 12323 0 0 0 1268 1 0 0 25 0 1 0 521623354 49152000 10138 33554432000 4194304 4352236 140733525019408 18446744073709551615 4328030 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2590/statm: 12000 10138 126 39 0 11080 0
[pid=2591] ppid=2586 vsize=72424 CPUtime=6.31 cores=7
/proc/2591/stat : 2591 (march_hi) R 2586 2586 2052 0 -1 4202496 38950 0 0 0 626 5 0 0 25 0 1 0 521623354 74162176 8430 33554432000 4194304 4331988 140735785846976 18446744073709551615 4226440 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2591/statm: 18106 8430 171 34 0 16548 0
Current children cumulated CPU time (s) 50.52
Current children cumulated vsize (KiB) 981764

[startup+25.5012 s]
/proc/loadavg: 10.41 10.22 9.76 13/189 2596
/proc/meminfo: memFree=29507916/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=309840 CPUtime=24.79 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 34517 0 0 0 2475 4 0 0 25 0 1 0 521623354 317276160 24132 33554432000 4194304 4587354 140734210868192 18446744073709551615 4365356 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 77460 24132 337 96 0 74022 0
[pid=2588] ppid=2586 vsize=132276 CPUtime=25.49 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 33955 0 0 0 2542 7 0 0 25 0 1 0 521623354 135450624 30180 33554432000 4194304 4785740 140734861304096 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 33069 30180 340 145 0 30114 0
[pid=2589] ppid=2586 vsize=483024 CPUtime=12.64 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 11668 0 0 0 1264 0 0 0 25 0 1 0 521623354 494616576 11565 33554432000 134512640 134534564 4289980672 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 120756 11565 95 6 0 119422 0
[pid=2590] ppid=2586 vsize=55788 CPUtime=25.49 cores=3
/proc/2590/stat : 2590 (lingeling) R 2586 2586 2052 0 -1 4202496 14270 0 0 0 2548 1 0 0 25 0 1 0 521623354 57126912 12085 33554432000 4194304 4352236 140733525019408 18446744073709551615 4328351 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2590/statm: 13947 12085 126 39 0 13027 0
[pid=2591] ppid=2586 vsize=74208 CPUtime=12.69 cores=7
/proc/2591/stat : 2591 (march_hi) R 2586 2586 2052 0 -1 4202496 39339 0 0 0 1264 5 0 0 25 0 1 0 521623354 75988992 8819 33554432000 4194304 4331988 140735785846976 18446744073709551615 4228235 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2591/statm: 18552 8819 174 34 0 16994 0
Current children cumulated CPU time (s) 101.1
Current children cumulated vsize (KiB) 1073460

[startup+51.1005 s]
/proc/loadavg: 10.41 10.23 9.78 11/188 2596
/proc/meminfo: memFree=29418464/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=319436 CPUtime=50.28 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 49245 0 0 0 5021 7 0 0 25 0 1 0 521623354 327102464 28582 33554432000 4194304 4587354 140734210868192 18446744073709551615 4365352 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 79859 28582 337 96 0 76421 0
[pid=2588] ppid=2586 vsize=211172 CPUtime=51.08 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 53163 0 0 0 5095 13 0 0 25 0 1 0 521623354 216240128 49388 33554432000 4194304 4785740 140734861304096 18446744073709551615 4287050 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 52793 49388 340 145 0 49838 0
[pid=2589] ppid=2586 vsize=483024 CPUtime=25.39 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 11668 0 0 0 2539 0 0 0 25 0 1 0 521623354 494616576 11565 33554432000 134512640 134534564 4289980672 18446744073709551615 134528302 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 120756 11565 95 6 0 119422 0
[pid=2590] ppid=2586 vsize=67232 CPUtime=51.09 cores=3

################
# More data... #
################

/proc/2591/statm: 29162 18814 175 34 0 27604 0
Current children cumulated CPU time (s) 3763.27
Current children cumulated vsize (KiB) 1835164

[startup+1002.3 s]
/proc/loadavg: 10.10 10.13 9.99 11/188 2655
/proc/meminfo: memFree=28675096/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=408040 CPUtime=1001.51 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 290495 0 0 0 100059 92 0 0 25 0 1 0 521623354 417832960 65916 33554432000 4194304 4587354 140734210868192 18446744073709551615 4365356 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 102010 65916 337 96 0 98572 0
[pid=2588] ppid=2586 vsize=618460 CPUtime=1002.3 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 150425 0 0 0 100183 47 0 0 25 0 1 0 521623354 633303040 146650 33554432000 4194304 4785740 140734861304096 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 154615 146650 340 145 0 151660 0
[pid=2589] ppid=2586 vsize=483024 CPUtime=498.39 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 11668 0 0 0 49838 1 0 0 25 0 1 0 521623354 494616576 11565 33554432000 134512640 134534564 4289980672 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 120756 11565 95 6 0 119422 0
[pid=2590] ppid=2586 vsize=181612 CPUtime=1002.32 cores=3
/proc/2590/stat : 2590 (lingeling) R 2586 2586 2052 0 -1 4202496 194205 0 0 0 100209 23 0 0 25 0 1 0 521623354 185970688 44564 33554432000 4194304 4352236 140733525019408 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2590/statm: 45403 44564 126 39 0 44483 0
[pid=2591] ppid=2586 vsize=136292 CPUtime=498.41 cores=7
/proc/2591/stat : 2591 (march_hi) R 2586 2586 2052 0 -1 4202496 49924 0 0 0 49830 11 0 0 25 0 1 0 521623354 139563008 19404 33554432000 4194304 4331988 140735785846976 18446744073709551615 4316990 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2591/statm: 34073 19404 175 34 0 32515 0
Current children cumulated CPU time (s) 4002.93
Current children cumulated vsize (KiB) 1845752

[startup+1062.3 s]
/proc/loadavg: 10.30 10.18 10.01 11/188 2656
/proc/meminfo: memFree=28654080/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=467608 CPUtime=1061.51 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 303499 0 0 0 106053 98 0 0 25 0 1 0 521623354 478830592 63795 33554432000 4194304 4587354 140734210868192 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 116902 63795 337 96 0 113464 0
[pid=2588] ppid=2586 vsize=627992 CPUtime=1062.31 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 152608 0 0 0 106183 48 0 0 25 0 1 0 521623354 643063808 148833 33554432000 4194304 4785740 140734861304096 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 156998 148833 340 145 0 154043 0
[pid=2589] ppid=2586 vsize=483024 CPUtime=528.23 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 11668 0 0 0 52822 1 0 0 25 0 1 0 521623354 494616576 11565 33554432000 134512640 134534564 4289980672 18446744073709551615 134528302 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 120756 11565 95 6 0 119422 0
[pid=2590] ppid=2586 vsize=148848 CPUtime=1062.32 cores=3
/proc/2590/stat : 2590 (lingeling) R 2586 2586 2052 0 -1 4202496 212639 0 0 0 106206 26 0 0 25 0 1 0 521623354 152420352 36406 33554432000 4194304 4352236 140733525019408 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2590/statm: 37212 36406 127 39 0 36292 0
[pid=2591] ppid=2586 vsize=136292 CPUtime=528.26 cores=7
/proc/2591/stat : 2591 (march_hi) R 2586 2586 2052 0 -1 4202496 50513 0 0 0 52814 12 0 0 25 0 1 0 521623354 139563008 19993 33554432000 4194304 4331988 140735785846976 18446744073709551615 4284383 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2591/statm: 34073 19993 175 34 0 32515 0
Current children cumulated CPU time (s) 4242.63
Current children cumulated vsize (KiB) 1882088

[startup+1122.3 s]
/proc/loadavg: 10.26 10.18 10.02 11/188 2658
/proc/meminfo: memFree=28658972/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=460176 CPUtime=1121.51 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 318716 0 0 0 112044 107 0 0 25 0 1 0 521623354 471220224 66070 33554432000 4194304 4587354 140734210868192 18446744073709551615 4244000 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 115044 66070 337 96 0 111606 0
[pid=2588] ppid=2586 vsize=641504 CPUtime=1122.31 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 155509 0 0 0 112183 48 0 0 25 0 1 0 521623354 656900096 151734 33554432000 4194304 4785740 140734861304096 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 160376 151734 340 145 0 157421 0
[pid=2589] ppid=2586 vsize=483024 CPUtime=558.05 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 11668 0 0 0 55804 1 0 0 25 0 1 0 521623354 494616576 11565 33554432000 134512640 134534564 4289980672 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 120756 11565 95 6 0 119422 0
[pid=2590] ppid=2586 vsize=132464 CPUtime=1122.32 cores=3
/proc/2590/stat : 2590 (lingeling) R 2586 2586 2052 0 -1 4202496 219807 0 0 0 112206 26 0 0 25 0 1 0 521623354 135643136 32310 33554432000 4194304 4352236 140733525019408 18446744073709551615 4329086 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2590/statm: 33116 32310 127 39 0 32196 0
[pid=2591] ppid=2586 vsize=136292 CPUtime=558.11 cores=7
/proc/2591/stat : 2591 (march_hi) R 2586 2586 2052 0 -1 4202496 51105 0 0 0 55799 12 0 0 25 0 1 0 521623354 139563008 20585 33554432000 4194304 4331988 140735785846976 18446744073709551615 4210448 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2591/statm: 34073 20585 175 34 0 32515 0
Current children cumulated CPU time (s) 4482.3
Current children cumulated vsize (KiB) 1871784

[startup+1182.3 s]
/proc/loadavg: 10.29 10.20 10.03 11/188 2660
/proc/meminfo: memFree=28659412/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=445440 CPUtime=1181.52 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 333708 0 0 0 118037 115 0 0 25 0 1 0 521623354 456130560 63795 33554432000 4194304 4587354 140734210868192 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 111360 63795 337 96 0 107922 0
[pid=2588] ppid=2586 vsize=652384 CPUtime=1182.31 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 158121 0 0 0 118182 49 0 0 25 0 1 0 521623354 668041216 154346 33554432000 4194304 4785740 140734861304096 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 163096 154346 340 145 0 160141 0
[pid=2589] ppid=2586 vsize=483024 CPUtime=587.92 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 11668 0 0 0 58791 1 0 0 25 0 1 0 521623354 494616576 11565 33554432000 134512640 134534564 4289980672 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 120756 11565 95 6 0 119422 0
[pid=2590] ppid=2586 vsize=142704 CPUtime=1182.32 cores=3
/proc/2590/stat : 2590 (lingeling) R 2586 2586 2052 0 -1 4202496 231071 0 0 0 118205 27 0 0 25 0 1 0 521623354 146128896 34870 33554432000 4194304 4352236 140733525019408 18446744073709551615 4329349 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2590/statm: 35676 34870 127 39 0 34756 0
[pid=2591] ppid=2586 vsize=136292 CPUtime=587.92 cores=7
/proc/2591/stat : 2591 (march_hi) R 2586 2586 2052 0 -1 4202496 51694 0 0 0 58780 12 0 0 25 0 1 0 521623354 139563008 21174 33554432000 4194304 4331988 140735785846976 18446744073709551615 4226485 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2591/statm: 34073 21174 175 34 0 32515 0
Current children cumulated CPU time (s) 4721.99
Current children cumulated vsize (KiB) 1878168



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 10.21 10.19 10.03 11/188 2660
/proc/meminfo: memFree=28610292/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=465924 CPUtime=1201.11 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 339476 0 0 0 119994 117 0 0 25 0 1 0 521623354 477106176 63795 33554432000 4194304 4587354 140734210868192 18446744073709551615 4238606 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 116481 63795 337 96 0 113043 0
[pid=2588] ppid=2586 vsize=655904 CPUtime=1201.92 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 158849 0 0 0 120143 49 0 0 25 0 1 0 521623354 671645696 155074 33554432000 4194304 4785740 140734861304096 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 163976 155074 340 145 0 161021 0
[pid=2589] ppid=2586 vsize=483024 CPUtime=597.63 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 11668 0 0 0 59762 1 0 0 25 0 1 0 521623354 494616576 11565 33554432000 134512640 134534564 4289980672 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 120756 11565 95 6 0 119422 0
[pid=2590] ppid=2586 vsize=169328 CPUtime=1201.93 cores=3
/proc/2590/stat : 2590 (lingeling) R 2586 2586 2052 0 -1 4202496 237727 0 0 0 120165 28 0 0 25 0 1 0 521623354 173391872 41526 33554432000 4194304 4352236 140733525019408 18446744073709551615 4328380 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2590/statm: 42332 41526 127 39 0 41412 0
[pid=2591] ppid=2586 vsize=136292 CPUtime=597.7 cores=7
/proc/2591/stat : 2591 (march_hi) R 2586 2586 2052 0 -1 4202496 51888 0 0 0 59758 12 0 0 25 0 1 0 521623354 139563008 21368 33554432000 4194304 4331988 140735785846976 18446744073709551615 4226496 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2591/statm: 34073 21368 175 34 0 32515 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 1928796

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.9 s]
/proc/loadavg: 10.21 10.19 10.03 11/188 2660
/proc/meminfo: memFree=28610292/32951124 swapFree=67110188/67111528
[pid=2586] ppid=2584 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2586/stat : 2586 (ppfolio) S 2584 2586 2052 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 521623354 18763776 389 33554432000 4194304 4287625 140735134262000 18446744073709551615 222921797135 0 0 4096 65536 4656388317217030143 0 0 17 5 0 0 0
/proc/2586/statm: 4581 389 341 23 0 91 0
[pid=2587] ppid=2586 vsize=465924 CPUtime=1201.11 cores=1
/proc/2587/stat : 2587 (cryptominisat) R 2586 2586 2052 0 -1 4202496 339476 0 0 0 119994 117 0 0 25 0 1 0 521623354 477106176 63795 33554432000 4194304 4587354 140734210868192 18446744073709551615 4238606 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2587/statm: 116481 63795 337 96 0 113043 0
[pid=2588] ppid=2586 vsize=655904 CPUtime=1201.92 cores=5
/proc/2588/stat : 2588 (clasp) R 2586 2586 2052 0 -1 4202496 158849 0 0 0 120143 49 0 0 25 0 1 0 521623354 671645696 155074 33554432000 4194304 4785740 140734861304096 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2588/statm: 163976 155074 340 145 0 161021 0
[pid=2589] ppid=2586 vsize=483024 CPUtime=597.63 cores=7
/proc/2589/stat : 2589 (TNM) R 2586 2586 2052 0 -1 4202496 11668 0 0 0 59762 1 0 0 25 0 1 0 521623354 494616576 11565 33554432000 134512640 134534564 4289980672 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2589/statm: 120756 11565 95 6 0 119422 0
[pid=2590] ppid=2586 vsize=169328 CPUtime=1201.93 cores=3
/proc/2590/stat : 2590 (lingeling) R 2586 2586 2052 0 -1 4202496 237727 0 0 0 120165 28 0 0 25 0 1 0 521623354 173391872 41526 33554432000 4194304 4352236 140733525019408 18446744073709551615 4328380 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2590/statm: 42332 41526 127 39 0 41412 0
[pid=2591] ppid=2586 vsize=136292 CPUtime=597.7 cores=7
/proc/2591/stat : 2591 (march_hi) R 2586 2586 2052 0 -1 4202496 51888 0 0 0 59758 12 0 0 25 0 1 0 521623354 139563008 21368 33554432000 4194304 4331988 140735785846976 18446744073709551615 4226496 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2591/statm: 34073 21368 175 34 0 32515 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 1928796

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 2586 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=2999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=1999
# CPU time returned by wait4() is 0.004998
# while last known CPU time is 4800.29
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.92
CPU time (s): 4800.29
CPU user time (s): 4798.22
CPU system time (s): 2.07
CPU usage (%): 399.385
Max. virtual memory (cumulated for all children) (KiB): 2186764

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.002999
system time used= 0.001999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 522
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 284
involuntary context switches= 6

runsolver used 1.79973 second user time and 4.76427 second system time

The end

Launcher Data 

Begin job on node109 at 2011-04-29 20:03:46
IDJOB=3251567
IDBENCH=88226
IDSOLVER=1612
FILE ID=node109/3251567-1304100226
RUNJOBID= node109-1304096344-2071
PBS_JOBID= 13170607
Free space on /tmp= 72364 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m52-q1041.25-n50-p15-s99099953.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3251567-1304100226/watcher-3251567-1304100226 -o /tmp/evaluation-result-3251567-1304100226/solver-3251567-1304100226 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3251567-1304100226.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= dda220d08783aabf4e7a9e99eeeccae6
RANDOM SEED=1801633509

node109.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29881640 kB
Buffers:        726712 kB
Cached:        1021200 kB
SwapCached:        712 kB
Active:        1025764 kB
Inactive:      1652256 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29881640 kB
SwapTotal:    67111528 kB
SwapFree:     67110188 kB
Dirty:           16344 kB
Writeback:           0 kB
AnonPages:      929840 kB
Mapped:          16620 kB
Slab:           324972 kB
PageTables:       6944 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  1989512 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72352 MiB
End job on node109 at 2011-04-29 20:23:50