Trace number 3190777

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.31 1201.71

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v35000-c147000-S883457431-033-UNKNOWN.cnf
MD5SUM0d0abbf84f73b4226aa3bedf08367d2e
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark291.591
Satisfiable
(Un)Satisfiability was proved
Number of variables35000
Number of clauses147000
Sum of the clauses size441000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3147000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.00	Starting clasp 1.3.6 on core(s) 5
0.00/0.00	Starting TNM 2009 on core(s) 7
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.00	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3190777-1304035755/watcher-3190777-1304035755 -o /tmp/evaluation-result-3190777-1304035755/solver-3190777-1304035755 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3190777-1304035755.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.22 10.13 9.87 6/182 17877
/proc/meminfo: memFree=11861972/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 518 0 0 0 0 0 0 0 19 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0

[startup+1.01277 s]
/proc/loadavg: 10.22 10.13 9.87 6/182 17877
/proc/meminfo: memFree=11861972/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 518 0 0 0 0 0 0 0 19 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.11275 s]
/proc/loadavg: 10.22 10.13 9.87 6/182 17877
/proc/meminfo: memFree=11861972/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 518 0 0 0 0 0 0 0 19 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.20074 s]
/proc/loadavg: 10.22 10.13 9.87 6/182 17877
/proc/meminfo: memFree=11861972/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 518 0 0 0 0 0 0 0 19 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.31272 s]
/proc/loadavg: 10.22 10.13 9.87 6/182 17877
/proc/meminfo: memFree=11861972/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 518 0 0 0 0 0 0 0 19 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.51268 s]
/proc/loadavg: 10.22 10.13 9.87 6/182 17877
/proc/meminfo: memFree=11861972/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 518 0 0 0 0 0 0 0 19 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+3.10038 s]
/proc/loadavg: 10.20 10.13 9.87 11/188 17883
/proc/meminfo: memFree=11738816/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 518 0 0 0 0 0 0 0 18 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=94732 CPUtime=3.09 cores=1
/proc/17878/stat : 17878 (cryptominisat) R 17877 17877 16932 0 -1 4202496 13834 0 0 0 307 2 0 0 25 0 1 0 593806180 97005568 10446 33554432000 4194304 4587354 140736784878960 18446744073709551615 4236870 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17878/statm: 23683 10446 326 96 0 20245 0
[pid=17879] ppid=17877 vsize=35244 CPUtime=3.09 cores=5
/proc/17879/stat : 17879 (clasp) R 17877 17877 16932 0 -1 4202496 7996 0 0 0 308 1 0 0 25 0 1 0 593806180 36089856 6085 33554432000 4194304 4785740 140734531832128 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17879/statm: 8811 6085 340 145 0 5856 0
[pid=17880] ppid=17877 vsize=465988 CPUtime=1.98 cores=7
/proc/17880/stat : 17880 (TNM) R 17877 17877 16932 0 -1 4202496 5914 0 0 0 197 1 0 0 21 0 1 0 593806180 477171712 5811 33554432000 134512640 134534564 4294354976 18446744073709551615 134529640 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17880/statm: 116497 5811 95 6 0 115163 0
[pid=17881] ppid=17877 vsize=19360 CPUtime=3.09 cores=3
/proc/17881/stat : 17881 (lingeling) R 17877 17877 16932 0 -1 4202496 6330 0 0 0 309 0 0 0 25 0 1 0 593806180 19824640 3495 33554432000 4194304 4352236 140735517262896 18446744073709551615 4328047 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17881/statm: 4840 3495 127 39 0 3920 0
[pid=17882] ppid=17877 vsize=33184 CPUtime=1.09 cores=7
/proc/17882/stat : 17882 (march_hi) R 17877 17877 16932 0 -1 4202496 24638 0 0 0 106 3 0 0 25 0 1 0 593806180 33980416 5420 33554432000 4194304 4331988 140733293257232 18446744073709551615 4252204 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17882/statm: 8296 5420 141 34 0 6738 0
Current children cumulated CPU time (s) 12.34
Current children cumulated vsize (KiB) 666832

[startup+6.30078 s]
/proc/loadavg: 10.20 10.13 9.87 11/188 17883
/proc/meminfo: memFree=11700992/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=94732 CPUtime=6.29 cores=1
/proc/17878/stat : 17878 (cryptominisat) R 17877 17877 16932 0 -1 4202496 15505 0 0 0 626 3 0 0 25 0 1 0 593806180 97005568 12109 33554432000 4194304 4587354 140736784878960 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17878/statm: 23683 12109 326 96 0 20245 0
[pid=17879] ppid=17877 vsize=48004 CPUtime=6.29 cores=5
/proc/17879/stat : 17879 (clasp) R 17877 17877 16932 0 -1 4202496 11195 0 0 0 627 2 0 0 25 0 1 0 593806180 49156096 9284 33554432000 4194304 4785740 140734531832128 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17879/statm: 12001 9284 340 145 0 9046 0
[pid=17880] ppid=17877 vsize=465988 CPUtime=3.56 cores=7
/proc/17880/stat : 17880 (TNM) R 17877 17877 16932 0 -1 4202496 5914 0 0 0 355 1 0 0 25 0 1 0 593806180 477171712 5811 33554432000 134512640 134534564 4294354976 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17880/statm: 116497 5811 95 6 0 115163 0
[pid=17881] ppid=17877 vsize=19360 CPUtime=6.28 cores=3
/proc/17881/stat : 17881 (lingeling) R 17877 17877 16932 0 -1 4202496 6330 0 0 0 628 0 0 0 25 0 1 0 593806180 19824640 3495 33554432000 4194304 4352236 140735517262896 18446744073709551615 4327879 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17881/statm: 4840 3495 127 39 0 3920 0
[pid=17882] ppid=17877 vsize=76464 CPUtime=2.69 cores=7
/proc/17882/stat : 17882 (march_hi) R 17877 17877 16932 0 -1 4202496 52364 0 0 0 261 8 0 0 25 0 1 0 593806180 78299136 15856 33554432000 4194304 4331988 140733293257232 18446744073709551615 4299847 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17882/statm: 19116 15856 173 34 0 17558 0
Current children cumulated CPU time (s) 25.11
Current children cumulated vsize (KiB) 722872

[startup+12.7006 s]
/proc/loadavg: 10.17 10.12 9.87 11/188 17884
/proc/meminfo: memFree=11646812/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=64668 CPUtime=12.68 cores=1
/proc/17878/stat : 17878 (cryptominisat) R 17877 17877 16932 0 -1 4202496 20372 0 0 0 1263 5 0 0 25 0 1 0 593806180 66220032 10606 33554432000 4194304 4587354 140736784878960 18446744073709551615 4238297 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17878/statm: 16167 10606 327 96 0 12729 0
[pid=17879] ppid=17877 vsize=70964 CPUtime=12.69 cores=5
/proc/17879/stat : 17879 (clasp) R 17877 17877 16932 0 -1 4202496 16917 0 0 0 1266 3 0 0 25 0 1 0 593806180 72667136 15006 33554432000 4194304 4785740 140734531832128 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17879/statm: 17741 15006 340 145 0 14786 0
[pid=17880] ppid=17877 vsize=465988 CPUtime=6.72 cores=7
/proc/17880/stat : 17880 (TNM) R 17877 17877 16932 0 -1 4202496 5914 0 0 0 671 1 0 0 25 0 1 0 593806180 477171712 5811 33554432000 134512640 134534564 4294354976 18446744073709551615 134530593 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17880/statm: 116497 5811 95 6 0 115163 0
[pid=17881] ppid=17877 vsize=23492 CPUtime=12.69 cores=3
/proc/17881/stat : 17881 (lingeling) R 17877 17877 16932 0 -1 4202496 7874 0 0 0 1268 1 0 0 25 0 1 0 593806180 24055808 5039 33554432000 4194304 4352236 140735517262896 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17881/statm: 5873 5039 127 39 0 4953 0
[pid=17882] ppid=17877 vsize=78120 CPUtime=5.89 cores=7
/proc/17882/stat : 17882 (march_hi) R 17877 17877 16932 0 -1 4202496 52779 0 0 0 581 8 0 0 25 0 1 0 593806180 79994880 16271 33554432000 4194304 4331988 140733293257232 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17882/statm: 19530 16271 173 34 0 17972 0
Current children cumulated CPU time (s) 50.67
Current children cumulated vsize (KiB) 721556

[startup+25.5011 s]
/proc/loadavg: 10.14 10.12 9.88 11/188 17884
/proc/meminfo: memFree=11578872/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=67476 CPUtime=25.48 cores=1
/proc/17878/stat : 17878 (cryptominisat) R 17877 17877 16932 0 -1 4202496 50776 0 0 0 2537 11 0 0 25 0 1 0 593806180 69095424 10096 33554432000 4194304 4587354 140736784878960 18446744073709551615 4250443 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17878/statm: 16869 10096 337 96 0 13431 0
[pid=17879] ppid=17877 vsize=114508 CPUtime=25.49 cores=5
/proc/17879/stat : 17879 (clasp) R 17877 17877 16932 0 -1 4202496 27754 0 0 0 2543 6 0 0 25 0 1 0 593806180 117256192 25843 33554432000 4194304 4785740 140734531832128 18446744073709551615 4550368 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17879/statm: 28627 25843 340 145 0 25672 0
[pid=17880] ppid=17877 vsize=465988 CPUtime=13.09 cores=7
/proc/17880/stat : 17880 (TNM) R 17877 17877 16932 0 -1 4202496 5914 0 0 0 1308 1 0 0 25 0 1 0 593806180 477171712 5811 33554432000 134512640 134534564 4294354976 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17880/statm: 116497 5811 95 6 0 115163 0
[pid=17881] ppid=17877 vsize=25204 CPUtime=25.49 cores=3
/proc/17881/stat : 17881 (lingeling) R 17877 17877 16932 0 -1 4202496 8302 0 0 0 2548 1 0 0 25 0 1 0 593806180 25808896 5467 33554432000 4194304 4352236 140735517262896 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17881/statm: 6301 5467 127 39 0 5381 0
[pid=17882] ppid=17877 vsize=85012 CPUtime=12.25 cores=7
/proc/17882/stat : 17882 (march_hi) R 17877 17877 16932 0 -1 4202496 54492 0 0 0 1216 9 0 0 25 0 1 0 593806180 87052288 17984 33554432000 4194304 4331988 140733293257232 18446744073709551615 4288429 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17882/statm: 21253 17984 173 34 0 19695 0
Current children cumulated CPU time (s) 101.8
Current children cumulated vsize (KiB) 776512

[startup+51.1013 s]
/proc/loadavg: 10.23 10.14 9.89 11/188 17885
/proc/meminfo: memFree=11531584/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=380664 CPUtime=51.09 cores=1
/proc/17878/stat : 17878 (cryptominisat) R 17877 17877 16932 0 -1 4202496 90725 0 0 0 5087 22 0 0 25 0 1 0 593806180 389799936 21001 33554432000 4194304 4587354 140736784878960 18446744073709551615 4236726 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17878/statm: 95166 21001 337 96 0 91728 0
[pid=17879] ppid=17877 vsize=137256 CPUtime=51.09 cores=5
/proc/17879/stat : 17879 (clasp) R 17877 17877 16932 0 -1 4202496 33245 0 0 0 5101 8 0 0 25 0 1 0 593806180 140550144 31334 33554432000 4194304 4785740 140734531832128 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17879/statm: 34314 31334 340 145 0 31359 0
[pid=17880] ppid=17877 vsize=465988 CPUtime=25.82 cores=7
/proc/17880/stat : 17880 (TNM) R 17877 17877 16932 0 -1 4202496 5914 0 0 0 2581 1 0 0 25 0 1 0 593806180 477171712 5811 33554432000 134512640 134534564 4294354976 18446744073709551615 134528737 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17880/statm: 116497 5811 95 6 0 115163 0
[pid=17881] ppid=17877 vsize=26188 CPUtime=51.09 cores=3
/proc/17881/stat : 17881 (lingeling) R 17877 17877 16932 0 -1 4202496 8548 0 0 0 5108 1 0 0 25 0 1 0 593806180 26816512 5713 33554432000 4194304 4352236 140735517262896 18446744073709551615 4207874 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17881/statm: 6547 5713 127 39 0 5627 0
[pid=17882] ppid=17877 vsize=86596 CPUtime=24.98 cores=7
/proc/17882/stat : 17882 (march_hi) R 17877 17877 16932 0 -1 4202496 54919 0 0 0 2489 9 0 0 25 0 1 0 593806180 88674304 18411 33554432000 4194304 4331988 140733293257232 18446744073709551615 4228963 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17882/statm: 21649 18411 173 34 0 20091 0
Current children cumulated CPU time (s) 204.07
Current children cumulated vsize (KiB) 1115016

[startup+102.301 s]
/proc/loadavg: 10.22 10.14 9.90 11/188 17886
/proc/meminfo: memFree=11459280/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=157800 CPUtime=102.29 cores=1

################
# More data... #
################

/proc/17882/statm: 22936 19710 173 34 0 21378 0
Current children cumulated CPU time (s) 3764.08
Current children cumulated vsize (KiB) 1397392

[startup+1002.3 s]
/proc/loadavg: 5.11 5.41 7.06 6/177 17927
/proc/meminfo: memFree=12182468/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=398040 CPUtime=1002.32 cores=1
/proc/17878/stat : 17878 (cryptominisat) R 17877 17877 16932 0 -1 4202496 614322 0 0 0 99981 251 0 0 25 0 1 0 593806180 407592960 23295 33554432000 4194304 4587354 140736784878960 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17878/statm: 99510 23295 337 96 0 96072 0
[pid=17879] ppid=17877 vsize=408124 CPUtime=1002.29 cores=5
/proc/17879/stat : 17879 (clasp) R 17877 17877 16932 0 -1 4202496 99691 0 0 0 100199 30 0 0 25 0 1 0 593806180 417918976 97780 33554432000 4194304 4785740 140734531832128 18446744073709551615 4286953 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17879/statm: 102031 97780 340 145 0 99076 0
[pid=17880] ppid=17877 vsize=465988 CPUtime=498.84 cores=7
/proc/17880/stat : 17880 (TNM) R 17877 17877 16932 0 -1 4202496 5914 0 0 0 49883 1 0 0 25 0 1 0 593806180 477171712 5811 33554432000 134512640 134534564 4294354976 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17880/statm: 116497 5811 95 6 0 115163 0
[pid=17881] ppid=17877 vsize=31876 CPUtime=1002.29 cores=3
/proc/17881/stat : 17881 (lingeling) R 17877 17877 16932 0 -1 4202496 9970 0 0 0 100228 1 0 0 25 0 1 0 593806180 32641024 7135 33554432000 4194304 4352236 140735517262896 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17881/statm: 7969 7135 127 39 0 7049 0
[pid=17882] ppid=17877 vsize=91876 CPUtime=498.01 cores=7
/proc/17882/stat : 17882 (march_hi) R 17877 17877 16932 0 -1 4202496 56250 0 0 0 49791 10 0 0 25 0 1 0 593806180 94081024 19742 33554432000 4194304 4331988 140733293257232 18446744073709551615 4226474 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17882/statm: 22969 19742 173 34 0 21411 0
Current children cumulated CPU time (s) 4003.75
Current children cumulated vsize (KiB) 1414228

[startup+1062.3 s]
/proc/loadavg: 5.04 5.33 6.93 6/177 17929
/proc/meminfo: memFree=12154416/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=416660 CPUtime=1062.32 cores=1
/proc/17878/stat : 17878 (cryptominisat) R 17877 17877 16932 0 -1 4202496 634395 0 0 0 105968 264 0 0 25 0 1 0 593806180 426659840 30811 33554432000 4194304 4587354 140736784878960 18446744073709551615 4244611 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17878/statm: 104165 30811 337 96 0 100727 0
[pid=17879] ppid=17877 vsize=408612 CPUtime=1062.3 cores=5
/proc/17879/stat : 17879 (clasp) R 17877 17877 16932 0 -1 4202496 99796 0 0 0 106200 30 0 0 25 0 1 0 593806180 418418688 97885 33554432000 4194304 4785740 140734531832128 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17879/statm: 102153 97885 340 145 0 99198 0
[pid=17880] ppid=17877 vsize=465988 CPUtime=528.71 cores=7
/proc/17880/stat : 17880 (TNM) R 17877 17877 16932 0 -1 4202496 5914 0 0 0 52870 1 0 0 25 0 1 0 593806180 477171712 5811 33554432000 134512640 134534564 4294354976 18446744073709551615 134530186 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17880/statm: 116497 5811 95 6 0 115163 0
[pid=17881] ppid=17877 vsize=31876 CPUtime=1062.29 cores=3
/proc/17881/stat : 17881 (lingeling) R 17877 17877 16932 0 -1 4202496 9970 0 0 0 106228 1 0 0 25 0 1 0 593806180 32641024 7135 33554432000 4194304 4352236 140735517262896 18446744073709551615 4276793 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17881/statm: 7969 7135 127 39 0 7049 0
[pid=17882] ppid=17877 vsize=92008 CPUtime=527.81 cores=7
/proc/17882/stat : 17882 (march_hi) R 17877 17877 16932 0 -1 4202496 56278 0 0 0 52771 10 0 0 25 0 1 0 593806180 94216192 19770 33554432000 4194304 4331988 140733293257232 18446744073709551615 4263258 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17882/statm: 23002 19770 173 34 0 21444 0
Current children cumulated CPU time (s) 4243.43
Current children cumulated vsize (KiB) 1433468

[startup+1122.3 s]
/proc/loadavg: 5.07 5.28 6.81 6/177 17930
/proc/meminfo: memFree=12176220/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=412528 CPUtime=1122.33 cores=1
/proc/17878/stat : 17878 (cryptominisat) R 17877 17877 16932 0 -1 4202496 652510 0 0 0 111956 277 0 0 25 0 1 0 593806180 422428672 23441 33554432000 4194304 4587354 140736784878960 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17878/statm: 103132 23441 337 96 0 99694 0
[pid=17879] ppid=17877 vsize=413928 CPUtime=1122.3 cores=5
/proc/17879/stat : 17879 (clasp) R 17877 17877 16932 0 -1 4202496 101148 0 0 0 112199 31 0 0 25 0 1 0 593806180 423862272 99237 33554432000 4194304 4785740 140734531832128 18446744073709551615 4287204 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17879/statm: 103482 99237 340 145 0 100527 0
[pid=17880] ppid=17877 vsize=465988 CPUtime=558.51 cores=7
/proc/17880/stat : 17880 (TNM) R 17877 17877 16932 0 -1 4202496 5914 0 0 0 55850 1 0 0 25 0 1 0 593806180 477171712 5811 33554432000 134512640 134534564 4294354976 18446744073709551615 134528451 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17880/statm: 116497 5811 95 6 0 115163 0
[pid=17881] ppid=17877 vsize=31876 CPUtime=1122.29 cores=3
/proc/17881/stat : 17881 (lingeling) R 17877 17877 16932 0 -1 4202496 9970 0 0 0 112228 1 0 0 25 0 1 0 593806180 32641024 7135 33554432000 4194304 4352236 140735517262896 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17881/statm: 7969 7135 127 39 0 7049 0
[pid=17882] ppid=17877 vsize=92140 CPUtime=557.69 cores=7
/proc/17882/stat : 17882 (march_hi) R 17877 17877 16932 0 -1 4202496 56308 0 0 0 55759 10 0 0 25 0 1 0 593806180 94351360 19800 33554432000 4194304 4331988 140733293257232 18446744073709551615 4286594 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17882/statm: 23035 19800 173 34 0 21477 0
Current children cumulated CPU time (s) 4483.12
Current children cumulated vsize (KiB) 1434784

[startup+1182.3 s]
/proc/loadavg: 5.08 5.24 6.70 6/177 17932
/proc/meminfo: memFree=12125856/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=453612 CPUtime=1182.33 cores=1
/proc/17878/stat : 17878 (cryptominisat) R 17877 17877 16932 0 -1 4202496 673821 0 0 0 117941 292 0 0 25 0 1 0 593806180 464498688 31571 33554432000 4194304 4587354 140736784878960 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17878/statm: 113403 31571 337 96 0 109965 0
[pid=17879] ppid=17877 vsize=432064 CPUtime=1182.31 cores=5
/proc/17879/stat : 17879 (clasp) R 17877 17877 16932 0 -1 4202496 105641 0 0 0 118198 33 0 0 25 0 1 0 593806180 442433536 103730 33554432000 4194304 4785740 140734531832128 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17879/statm: 108016 103730 340 145 0 105061 0
[pid=17880] ppid=17877 vsize=465988 CPUtime=588.38 cores=7
/proc/17880/stat : 17880 (TNM) R 17877 17877 16932 0 -1 4202496 5914 0 0 0 58837 1 0 0 25 0 1 0 593806180 477171712 5811 33554432000 134512640 134534564 4294354976 18446744073709551615 134530158 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17880/statm: 116497 5811 95 6 0 115163 0
[pid=17881] ppid=17877 vsize=31876 CPUtime=1182.3 cores=3
/proc/17881/stat : 17881 (lingeling) R 17877 17877 16932 0 -1 4202496 9970 0 0 0 118229 1 0 0 25 0 1 0 593806180 32641024 7135 33554432000 4194304 4352236 140735517262896 18446744073709551615 4328022 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17881/statm: 7969 7135 127 39 0 7049 0
[pid=17882] ppid=17877 vsize=92272 CPUtime=587.5 cores=7
/proc/17882/stat : 17882 (march_hi) R 17877 17877 16932 0 -1 4202496 56338 0 0 0 58740 10 0 0 25 0 1 0 593806180 94486528 19830 33554432000 4194304 4331988 140733293257232 18446744073709551615 4240796 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17882/statm: 23068 19830 173 34 0 21510 0
Current children cumulated CPU time (s) 4722.82
Current children cumulated vsize (KiB) 1494136



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 5.05 5.23 6.66 6/177 17932
/proc/meminfo: memFree=12157476/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=427636 CPUtime=1201.73 cores=1
/proc/17878/stat : 17878 (cryptominisat) R 17877 17877 16932 0 -1 4202496 679219 0 0 0 119876 297 0 0 25 0 1 0 593806180 437899264 23484 33554432000 4194304 4587354 140736784878960 18446744073709551615 4243610 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17878/statm: 106909 23484 337 96 0 103471 0
[pid=17879] ppid=17877 vsize=432064 CPUtime=1201.72 cores=5
/proc/17879/stat : 17879 (clasp) R 17877 17877 16932 0 -1 4202496 105641 0 0 0 120139 33 0 0 25 0 1 0 593806180 442433536 103730 33554432000 4194304 4785740 140734531832128 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17879/statm: 108016 103730 340 145 0 105061 0
[pid=17880] ppid=17877 vsize=465988 CPUtime=598 cores=7
/proc/17880/stat : 17880 (TNM) R 17877 17877 16932 0 -1 4202496 5914 0 0 0 59799 1 0 0 25 0 1 0 593806180 477171712 5811 33554432000 134512640 134534564 4294354976 18446744073709551615 134530126 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17880/statm: 116497 5811 95 6 0 115163 0
[pid=17881] ppid=17877 vsize=31876 CPUtime=1201.7 cores=3
/proc/17881/stat : 17881 (lingeling) R 17877 17877 16932 0 -1 4202496 9970 0 0 0 120169 1 0 0 25 0 1 0 593806180 32641024 7135 33554432000 4194304 4352236 140735517262896 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17881/statm: 7969 7135 127 39 0 7049 0
[pid=17882] ppid=17877 vsize=92272 CPUtime=597.16 cores=7
/proc/17882/stat : 17882 (march_hi) R 17877 17877 16932 0 -1 4202496 56345 0 0 0 59706 10 0 0 25 0 1 0 593806180 94486528 19837 33554432000 4194304 4331988 140733293257232 18446744073709551615 4237171 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17882/statm: 23068 19837 173 34 0 21510 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 1468160

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 5.05 5.23 6.66 6/177 17932
/proc/meminfo: memFree=12157476/32950928 swapFree=67111524/67111528
[pid=17877] ppid=17875 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17877/stat : 17877 (ppfolio) S 17875 17877 16932 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 593806180 18763776 389 33554432000 4194304 4287625 140735443472144 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 1 0 0 0
/proc/17877/statm: 4581 389 341 23 0 91 0
[pid=17878] ppid=17877 vsize=427636 CPUtime=1201.73 cores=1
/proc/17878/stat : 17878 (cryptominisat) R 17877 17877 16932 0 -1 4202496 679219 0 0 0 119876 297 0 0 25 0 1 0 593806180 437899264 23484 33554432000 4194304 4587354 140736784878960 18446744073709551615 4243610 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17878/statm: 106909 23484 337 96 0 103471 0
[pid=17879] ppid=17877 vsize=432064 CPUtime=1201.72 cores=5
/proc/17879/stat : 17879 (clasp) R 17877 17877 16932 0 -1 4202496 105641 0 0 0 120139 33 0 0 25 0 1 0 593806180 442433536 103730 33554432000 4194304 4785740 140734531832128 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17879/statm: 108016 103730 340 145 0 105061 0
[pid=17880] ppid=17877 vsize=465988 CPUtime=598 cores=7
/proc/17880/stat : 17880 (TNM) R 17877 17877 16932 0 -1 4202496 5914 0 0 0 59799 1 0 0 25 0 1 0 593806180 477171712 5811 33554432000 134512640 134534564 4294354976 18446744073709551615 134530126 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17880/statm: 116497 5811 95 6 0 115163 0
[pid=17881] ppid=17877 vsize=31876 CPUtime=1201.7 cores=3
/proc/17881/stat : 17881 (lingeling) R 17877 17877 16932 0 -1 4202496 9970 0 0 0 120169 1 0 0 25 0 1 0 593806180 32641024 7135 33554432000 4194304 4352236 140735517262896 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17881/statm: 7969 7135 127 39 0 7049 0
[pid=17882] ppid=17877 vsize=92272 CPUtime=597.16 cores=7
/proc/17882/stat : 17882 (march_hi) R 17877 17877 16932 0 -1 4202496 56345 0 0 0 59706 10 0 0 25 0 1 0 593806180 94486528 19837 33554432000 4194304 4331988 140733293257232 18446744073709551615 4237171 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17882/statm: 23068 19837 173 34 0 21510 0
Current children cumulated CPU time (s) 4800.31
Current children cumulated vsize (KiB) 1468160

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 17877 and gives
#  childrusage.ru_utime.tv_sec=1201
#  childrusage.ru_utime.tv_usec=698314
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=22996
# CPU time returned by wait4() is 1201.72
# while last known CPU time is 4800.31
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.31
CPU user time (s): 4796.89
CPU system time (s): 3.42
CPU usage (%): 399.456
Max. virtual memory (cumulated for all children) (KiB): 1727804

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1201.7
system time used= 0.022996
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 10491
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 242
involuntary context switches= 1971

runsolver used 1.72474 second user time and 4.83326 second system time

The end

Launcher Data 

Begin job on node115 at 2011-04-29 02:09:15
IDJOB=3190777
IDBENCH=82508
IDSOLVER=1612
FILE ID=node115/3190777-1304035755
RUNJOBID= node115-1304032879-17473
PBS_JOBID= 13170337
Free space on /tmp= 72704 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v35000-c147000-S883457431-033-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3190777-1304035755/watcher-3190777-1304035755 -o /tmp/evaluation-result-3190777-1304035755/solver-3190777-1304035755 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3190777-1304035755.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 0d0abbf84f73b4226aa3bedf08367d2e
RANDOM SEED=415164322

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      11862252 kB
Buffers:       2315096 kB
Cached:       17246092 kB
SwapCached:          4 kB
Active:        8518196 kB
Inactive:     12010892 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      11862252 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           12860 kB
Writeback:           0 kB
AnonPages:      967668 kB
Mapped:          16944 kB
Slab:           492492 kB
PageTables:       6988 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  2972896 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72720 MiB
End job on node115 at 2011-04-29 02:29:19