Trace number 3191323

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.27 1201.91

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v90-c8010-S1897189325-062.UNKNOWN.cnf
MD5SUM28731ba8f9c5274ff0cd070fd7aeff45
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables90
Number of clauses8010
Sum of the clauses size56070
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 58010

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.00	Starting clasp 1.3.6 on core(s) 5
0.00/0.00	Starting TNM 2009 on core(s) 7
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.00	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3191323-1304078530/watcher-3191323-1304078530 -o /tmp/evaluation-result-3191323-1304078530/solver-3191323-1304078530 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3191323-1304078530.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.11 7.86 7.88 6/182 17698
/proc/meminfo: memFree=17888376/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) R 17696 17698 17121 0 -1 4202496 522 0 0 0 0 0 0 0 24 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 0 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0

[startup+0.921164 s]
/proc/loadavg: 7.11 7.86 7.88 6/182 17698
/proc/meminfo: memFree=17888376/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 22 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.00015 s]
/proc/loadavg: 7.11 7.86 7.88 6/182 17698
/proc/meminfo: memFree=17888376/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 22 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.12113 s]
/proc/loadavg: 7.11 7.86 7.88 6/182 17698
/proc/meminfo: memFree=17888376/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 20 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.20011 s]
/proc/loadavg: 7.11 7.86 7.88 6/182 17698
/proc/meminfo: memFree=17888376/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 20 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50106 s]
/proc/loadavg: 7.11 7.86 7.88 6/182 17698
/proc/meminfo: memFree=17888376/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 16 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+3.10678 s]
/proc/loadavg: 7.34 7.89 7.89 11/188 17704
/proc/meminfo: memFree=17848364/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=708396 CPUtime=3.09 cores=1
/proc/17699/stat : 17699 (cryptominisat) R 17698 17698 17121 0 -1 4202496 6039 0 0 0 309 0 0 0 25 0 1 0 519455650 725397504 3464 33554432000 4194304 4587354 140736003094400 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17699/statm: 177099 3464 336 96 0 173661 0
[pid=17700] ppid=17698 vsize=16156 CPUtime=3.1 cores=5
/proc/17700/stat : 17700 (clasp) R 17698 17698 17121 0 -1 4202496 1478 0 0 0 310 0 0 0 25 0 1 0 519455650 16543744 1369 33554432000 4194304 4785740 140737324327088 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17700/statm: 4039 1369 341 145 0 1084 0
[pid=17701] ppid=17698 vsize=456768 CPUtime=1.98 cores=7
/proc/17701/stat : 17701 (TNM) R 17698 17698 17121 0 -1 4202496 597 0 0 0 198 0 0 0 21 0 1 0 519455650 467730432 493 33554432000 134512640 134534564 4291879840 18446744073709551615 134528877 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17701/statm: 114192 493 95 6 0 112858 0
[pid=17702] ppid=17698 vsize=7620 CPUtime=3.1 cores=3
/proc/17702/stat : 17702 (lingeling) R 17698 17698 17121 0 -1 4202496 1570 0 0 0 310 0 0 0 25 0 1 0 519455650 7802880 1099 33554432000 4194304 4352236 140736874418224 18446744073709551615 4328345 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17702/statm: 1905 1099 127 39 0 985 0
[pid=17703] ppid=17698 vsize=8544 CPUtime=1.1 cores=7
/proc/17703/stat : 17703 (march_hi) R 17698 17698 17121 0 -1 4202496 1828 0 0 0 110 0 0 0 25 0 1 0 519455650 8749056 604 33554432000 4194304 4331988 140735605524912 18446744073709551615 4239564 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17703/statm: 2136 604 174 34 0 578 0
Current children cumulated CPU time (s) 12.37
Current children cumulated vsize (KiB) 1215808

[startup+6.30122 s]
/proc/loadavg: 7.56 7.93 7.90 11/188 17704
/proc/meminfo: memFree=17841048/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=36504 CPUtime=6.3 cores=1
/proc/17699/stat : 17699 (cryptominisat) R 17698 17698 17121 0 -1 4202496 7908 0 0 0 629 1 0 0 25 0 1 0 519455650 37380096 3523 33554432000 4194304 4587354 140736003094400 18446744073709551615 216634211996 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17699/statm: 9126 3523 336 96 0 5688 0
[pid=17700] ppid=17698 vsize=17560 CPUtime=6.3 cores=5
/proc/17700/stat : 17700 (clasp) R 17698 17698 17121 0 -1 4202496 1791 0 0 0 630 0 0 0 25 0 1 0 519455650 17981440 1682 33554432000 4194304 4785740 140737324327088 18446744073709551615 4550365 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17700/statm: 4390 1682 341 145 0 1435 0
[pid=17701] ppid=17698 vsize=456768 CPUtime=3.56 cores=7
/proc/17701/stat : 17701 (TNM) R 17698 17698 17121 0 -1 4202496 597 0 0 0 356 0 0 0 25 0 1 0 519455650 467730432 493 33554432000 134512640 134534564 4291879840 18446744073709551615 134528955 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17701/statm: 114192 493 95 6 0 112858 0
[pid=17702] ppid=17698 vsize=7912 CPUtime=6.3 cores=3
/proc/17702/stat : 17702 (lingeling) R 17698 17698 17121 0 -1 4202496 1912 0 0 0 630 0 0 0 25 0 1 0 519455650 8101888 1172 33554432000 4194304 4352236 140736874418224 18446744073709551615 4328068 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17702/statm: 1978 1172 127 39 0 1058 0
[pid=17703] ppid=17698 vsize=9108 CPUtime=2.7 cores=7
/proc/17703/stat : 17703 (march_hi) R 17698 17698 17121 0 -1 4202496 1939 0 0 0 270 0 0 0 25 0 1 0 519455650 9326592 715 33554432000 4194304 4331988 140735605524912 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17703/statm: 2277 715 174 34 0 719 0
Current children cumulated CPU time (s) 25.16
Current children cumulated vsize (KiB) 546176

[startup+12.7011 s]
/proc/loadavg: 7.75 7.96 7.91 11/188 17704
/proc/meminfo: memFree=17821844/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=44956 CPUtime=12.68 cores=1
/proc/17699/stat : 17699 (cryptominisat) R 17698 17698 17121 0 -1 4202496 12895 0 0 0 1267 1 0 0 25 0 1 0 519455650 46034944 5380 33554432000 4194304 4587354 140736003094400 18446744073709551615 4238392 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17699/statm: 11239 5380 336 96 0 7801 0
[pid=17700] ppid=17698 vsize=18948 CPUtime=12.69 cores=5
/proc/17700/stat : 17700 (clasp) R 17698 17698 17121 0 -1 4202496 2135 0 0 0 1269 0 0 0 25 0 1 0 519455650 19402752 2026 33554432000 4194304 4785740 140737324327088 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17700/statm: 4737 2026 341 145 0 1782 0
[pid=17701] ppid=17698 vsize=456768 CPUtime=6.73 cores=7
/proc/17701/stat : 17701 (TNM) R 17698 17698 17121 0 -1 4202496 597 0 0 0 673 0 0 0 25 0 1 0 519455650 467730432 493 33554432000 134512640 134534564 4291879840 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17701/statm: 114192 493 95 6 0 112858 0
[pid=17702] ppid=17698 vsize=10088 CPUtime=12.69 cores=3
/proc/17702/stat : 17702 (lingeling) R 17698 17698 17121 0 -1 4202496 3064 0 0 0 1269 0 0 0 25 0 1 0 519455650 10330112 1716 33554432000 4194304 4352236 140736874418224 18446744073709551615 4328000 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17702/statm: 2522 1716 127 39 0 1602 0
[pid=17703] ppid=17698 vsize=9488 CPUtime=5.9 cores=7
/proc/17703/stat : 17703 (march_hi) R 17698 17698 17121 0 -1 4202496 2037 0 0 0 590 0 0 0 25 0 1 0 519455650 9715712 813 33554432000 4194304 4331988 140735605524912 18446744073709551615 4239602 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17703/statm: 2372 813 175 34 0 814 0
Current children cumulated CPU time (s) 50.69
Current children cumulated vsize (KiB) 558572

[startup+25.5008 s]
/proc/loadavg: 8.17 8.05 7.94 11/188 17705
/proc/meminfo: memFree=17804064/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=50028 CPUtime=25.49 cores=1
/proc/17699/stat : 17699 (cryptominisat) R 17698 17698 17121 0 -1 4202496 19384 0 0 0 2546 3 0 0 25 0 1 0 519455650 51228672 6958 33554432000 4194304 4587354 140736003094400 18446744073709551615 4243819 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17699/statm: 12507 6958 336 96 0 9069 0
[pid=17700] ppid=17698 vsize=20804 CPUtime=25.49 cores=5
/proc/17700/stat : 17700 (clasp) R 17698 17698 17121 0 -1 4202496 2543 0 0 0 2549 0 0 0 25 0 1 0 519455650 21303296 2434 33554432000 4194304 4785740 140737324327088 18446744073709551615 4287053 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17700/statm: 5201 2434 341 145 0 2246 0
[pid=17701] ppid=17698 vsize=456768 CPUtime=13.11 cores=7
/proc/17701/stat : 17701 (TNM) R 17698 17698 17121 0 -1 4202496 597 0 0 0 1311 0 0 0 25 0 1 0 519455650 467730432 493 33554432000 134512640 134534564 4291879840 18446744073709551615 134528661 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17701/statm: 114192 493 95 6 0 112858 0
[pid=17702] ppid=17698 vsize=11016 CPUtime=25.49 cores=3
/proc/17702/stat : 17702 (lingeling) R 17698 17698 17121 0 -1 4202496 4048 0 0 0 2549 0 0 0 25 0 1 0 519455650 11280384 1932 33554432000 4194304 4352236 140736874418224 18446744073709551615 4328016 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17702/statm: 2754 1932 127 39 0 1834 0
[pid=17703] ppid=17698 vsize=10340 CPUtime=12.25 cores=7
/proc/17703/stat : 17703 (march_hi) R 17698 17698 17121 0 -1 4202496 2231 0 0 0 1225 0 0 0 25 0 1 0 519455650 10588160 1007 33554432000 4194304 4331988 140735605524912 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17703/statm: 2585 1007 175 34 0 1027 0
Current children cumulated CPU time (s) 101.83
Current children cumulated vsize (KiB) 567280

[startup+51.1003 s]
/proc/loadavg: 8.79 8.20 7.99 11/188 17705
/proc/meminfo: memFree=17772628/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=56864 CPUtime=51.09 cores=1
/proc/17699/stat : 17699 (cryptominisat) R 17698 17698 17121 0 -1 4202496 28351 0 0 0 5104 5 0 0 25 0 1 0 519455650 58228736 8506 33554432000 4194304 4587354 140736003094400 18446744073709551615 4239304 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17699/statm: 14216 8506 336 96 0 10778 0
[pid=17700] ppid=17698 vsize=23544 CPUtime=51.09 cores=5
/proc/17700/stat : 17700 (clasp) R 17698 17698 17121 0 -1 4202496 3153 0 0 0 5109 0 0 0 25 0 1 0 519455650 24109056 3044 33554432000 4194304 4785740 140737324327088 18446744073709551615 4549814 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17700/statm: 5886 3044 341 145 0 2931 0
[pid=17701] ppid=17698 vsize=456768 CPUtime=25.82 cores=7
/proc/17701/stat : 17701 (TNM) R 17698 17698 17121 0 -1 4202496 597 0 0 0 2582 0 0 0 25 0 1 0 519455650 467730432 493 33554432000 134512640 134534564 4291879840 18446744073709551615 134528979 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17701/statm: 114192 493 95 6 0 112858 0
[pid=17702] ppid=17698 vsize=11404 CPUtime=51.08 cores=3
/proc/17702/stat : 17702 (lingeling) R 17698 17698 17121 0 -1 4202496 6113 0 0 0 5108 0 0 0 25 0 1 0 519455650 11677696 2045 33554432000 4194304 4352236 140736874418224 18446744073709551615 4328377 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17702/statm: 2851 2045 127 39 0 1931 0
[pid=17703] ppid=17698 vsize=12256 CPUtime=24.99 cores=7
/proc/17703/stat : 17703 (march_hi) R 17698 17698 17121 0 -1 4202496 2600 0 0 0 2499 0 0 0 25 0 1 0 519455650 12550144 1376 33554432000 4194304 4331988 140735605524912 18446744073709551615 4299299 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17703/statm: 3064 1376 175 34 0 1506 0
Current children cumulated CPU time (s) 204.07
Current children cumulated vsize (KiB) 579160

[startup+102.3 s]
/proc/loadavg: 9.95 8.61 8.14 12/189 17710
/proc/meminfo: memFree=17709108/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=63780 CPUtime=102.28 cores=1

################
# More data... #
################

/proc/17703/statm: 17220 12886 175 34 0 15662 0
Current children cumulated CPU time (s) 3523.59
Current children cumulated vsize (KiB) 1377156

[startup+942.301 s]
/proc/loadavg: 10.18 10.07 9.33 11/188 17732
/proc/meminfo: memFree=17536784/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=84388 CPUtime=942.3 cores=1
/proc/17699/stat : 17699 (cryptominisat) R 17698 17698 17121 0 -1 4202496 143735 0 0 0 94175 55 0 0 25 0 1 0 519455650 86413312 15828 33554432000 4194304 4587354 140736003094400 18446744073709551615 4238568 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17699/statm: 21097 15828 336 96 0 17659 0
[pid=17700] ppid=17698 vsize=46684 CPUtime=941.51 cores=5
/proc/17700/stat : 17700 (clasp) R 17698 17698 17121 0 -1 4202496 8429 0 0 0 94150 1 0 0 25 0 1 0 519455650 47804416 8142 33554432000 4194304 4785740 140737324327088 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17700/statm: 11671 8142 341 145 0 8716 0
[pid=17701] ppid=17698 vsize=456768 CPUtime=469.04 cores=7
/proc/17701/stat : 17701 (TNM) R 17698 17698 17121 0 -1 4202496 597 0 0 0 46904 0 0 0 25 0 1 0 519455650 467730432 493 33554432000 134512640 134534564 4291879840 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17701/statm: 114192 493 95 6 0 112858 0
[pid=17702] ppid=17698 vsize=31624 CPUtime=942.28 cores=3
/proc/17702/stat : 17702 (lingeling) R 17698 17698 17121 0 -1 4202496 41282 0 0 0 94223 5 0 0 25 0 1 0 519455650 32382976 7100 33554432000 4194304 4352236 140736874418224 18446744073709551615 4329000 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17702/statm: 7906 7100 127 39 0 6986 0
[pid=17703] ppid=17698 vsize=68880 CPUtime=468.13 cores=7
/proc/17703/stat : 17703 (march_hi) R 17698 17698 17121 0 -1 4202496 14802 0 0 0 46806 7 0 0 25 0 1 0 519455650 70533120 13578 33554432000 4194304 4331988 140735605524912 18446744073709551615 4210430 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17703/statm: 17220 13578 175 34 0 15662 0
Current children cumulated CPU time (s) 3763.26
Current children cumulated vsize (KiB) 706668

[startup+1002.3 s]
/proc/loadavg: 10.11 10.07 9.37 11/188 17734
/proc/meminfo: memFree=17516512/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=828604 CPUtime=1002.3 cores=1
/proc/17699/stat : 17699 (cryptominisat) R 17698 17698 17121 0 -1 4202496 147498 0 0 0 100174 56 0 0 25 0 1 0 519455650 848490496 15666 33554432000 4194304 4587354 140736003094400 18446744073709551615 4238491 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17699/statm: 207151 15666 336 96 0 203713 0
[pid=17700] ppid=17698 vsize=46876 CPUtime=1001.51 cores=5
/proc/17700/stat : 17700 (clasp) R 17698 17698 17121 0 -1 4202496 8488 0 0 0 100150 1 0 0 25 0 1 0 519455650 48001024 8201 33554432000 4194304 4785740 140737324327088 18446744073709551615 4286919 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17700/statm: 11719 8201 341 145 0 8764 0
[pid=17701] ppid=17698 vsize=456768 CPUtime=498.83 cores=7
/proc/17701/stat : 17701 (TNM) R 17698 17698 17121 0 -1 4202496 597 0 0 0 49883 0 0 0 25 0 1 0 519455650 467730432 493 33554432000 134512640 134534564 4291879840 18446744073709551615 134528842 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17701/statm: 114192 493 95 6 0 112858 0
[pid=17702] ppid=17698 vsize=36488 CPUtime=1002.28 cores=3
/proc/17702/stat : 17702 (lingeling) R 17698 17698 17121 0 -1 4202496 43778 0 0 0 100222 6 0 0 25 0 1 0 519455650 37363712 8316 33554432000 4194304 4352236 140736874418224 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17702/statm: 9122 8316 127 39 0 8202 0
[pid=17703] ppid=17698 vsize=68880 CPUtime=498.02 cores=7
/proc/17703/stat : 17703 (march_hi) R 17698 17698 17121 0 -1 4202496 15663 0 0 0 49794 8 0 0 25 0 1 0 519455650 70533120 14439 33554432000 4194304 4331988 140735605524912 18446744073709551615 4210418 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17703/statm: 17220 14439 175 34 0 15662 0
Current children cumulated CPU time (s) 4002.94
Current children cumulated vsize (KiB) 1455940

[startup+1062.3 s]
/proc/loadavg: 10.12 10.08 9.42 11/188 17735
/proc/meminfo: memFree=17501216/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=130512 CPUtime=1062.3 cores=1
/proc/17699/stat : 17699 (cryptominisat) R 17698 17698 17121 0 -1 4202496 149722 0 0 0 106173 57 0 0 25 0 1 0 519455650 133644288 13954 33554432000 4194304 4587354 140736003094400 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17699/statm: 32628 13954 336 96 0 29190 0
[pid=17700] ppid=17698 vsize=47444 CPUtime=1061.52 cores=5
/proc/17700/stat : 17700 (clasp) R 17698 17698 17121 0 -1 4202496 8772 0 0 0 106151 1 0 0 25 0 1 0 519455650 48582656 8341 33554432000 4194304 4785740 140737324327088 18446744073709551615 4286953 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17700/statm: 11861 8341 341 145 0 8906 0
[pid=17701] ppid=17698 vsize=456768 CPUtime=528.71 cores=7
/proc/17701/stat : 17701 (TNM) R 17698 17698 17121 0 -1 4202496 597 0 0 0 52871 0 0 0 25 0 1 0 519455650 467730432 493 33554432000 134512640 134534564 4291879840 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17701/statm: 114192 493 95 6 0 112858 0
[pid=17702] ppid=17698 vsize=31884 CPUtime=1062.29 cores=3
/proc/17702/stat : 17702 (lingeling) R 17698 17698 17121 0 -1 4202496 45187 0 0 0 106223 6 0 0 25 0 1 0 519455650 32649216 7165 33554432000 4194304 4352236 140736874418224 18446744073709551615 4328208 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17702/statm: 7971 7165 127 39 0 7051 0
[pid=17703] ppid=17698 vsize=68880 CPUtime=527.81 cores=7
/proc/17703/stat : 17703 (march_hi) R 17698 17698 17121 0 -1 4202496 16484 0 0 0 52773 8 0 0 25 0 1 0 519455650 70533120 15260 33554432000 4194304 4331988 140735605524912 18446744073709551615 4210416 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17703/statm: 17220 15260 175 34 0 15662 0
Current children cumulated CPU time (s) 4242.63
Current children cumulated vsize (KiB) 753812

[startup+1122.3 s]
/proc/loadavg: 10.16 10.10 9.46 11/188 17737
/proc/meminfo: memFree=17486412/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=813352 CPUtime=1122.3 cores=1
/proc/17699/stat : 17699 (cryptominisat) R 17698 17698 17121 0 -1 4202496 153875 0 0 0 112171 59 0 0 25 0 1 0 519455650 832872448 13935 33554432000 4194304 4587354 140736003094400 18446744073709551615 4239304 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17699/statm: 203338 13935 336 96 0 199900 0
[pid=17700] ppid=17698 vsize=47828 CPUtime=1121.51 cores=5
/proc/17700/stat : 17700 (clasp) R 17698 17698 17121 0 -1 4202496 8910 0 0 0 112150 1 0 0 25 0 1 0 519455650 48975872 8431 33554432000 4194304 4785740 140737324327088 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17700/statm: 11957 8431 341 145 0 9002 0
[pid=17701] ppid=17698 vsize=456768 CPUtime=558.52 cores=7
/proc/17701/stat : 17701 (TNM) R 17698 17698 17121 0 -1 4202496 597 0 0 0 55852 0 0 0 25 0 1 0 519455650 467730432 493 33554432000 134512640 134534564 4291879840 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17701/statm: 114192 493 95 6 0 112858 0
[pid=17702] ppid=17698 vsize=32140 CPUtime=1122.28 cores=3
/proc/17702/stat : 17702 (lingeling) R 17698 17698 17121 0 -1 4202496 46531 0 0 0 112222 6 0 0 25 0 1 0 519455650 32911360 7229 33554432000 4194304 4352236 140736874418224 18446744073709551615 4328100 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17702/statm: 8035 7229 127 39 0 7115 0
[pid=17703] ppid=17698 vsize=98348 CPUtime=557.67 cores=7
/proc/17703/stat : 17703 (march_hi) R 17698 17698 17121 0 -1 4202496 17360 0 0 0 55758 9 0 0 25 0 1 0 519455650 100708352 16136 33554432000 4194304 4331988 140735605524912 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17703/statm: 24587 16136 175 34 0 23029 0
Current children cumulated CPU time (s) 4482.28
Current children cumulated vsize (KiB) 1466760

[startup+1182.31 s]
/proc/loadavg: 10.18 10.11 9.50 11/188 17738
/proc/meminfo: memFree=17459960/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=85536 CPUtime=1182.31 cores=1
/proc/17699/stat : 17699 (cryptominisat) R 17698 17698 17121 0 -1 4202496 161793 0 0 0 118169 62 0 0 25 0 1 0 519455650 87588864 17574 33554432000 4194304 4587354 140736003094400 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17699/statm: 21384 17574 336 96 0 17946 0
[pid=17700] ppid=17698 vsize=47924 CPUtime=1181.52 cores=5
/proc/17700/stat : 17700 (clasp) R 17698 17698 17121 0 -1 4202496 8994 0 0 0 118151 1 0 0 25 0 1 0 519455650 49074176 8467 33554432000 4194304 4785740 140737324327088 18446744073709551615 4286953 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17700/statm: 11981 8467 341 145 0 9026 0
[pid=17701] ppid=17698 vsize=456768 CPUtime=588.37 cores=7
/proc/17701/stat : 17701 (TNM) R 17698 17698 17121 0 -1 4202496 597 0 0 0 58837 0 0 0 25 0 1 0 519455650 467730432 493 33554432000 134512640 134534564 4291879840 18446744073709551615 134529640 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17701/statm: 114192 493 95 6 0 112858 0
[pid=17702] ppid=17698 vsize=33676 CPUtime=1182.28 cores=3
/proc/17702/stat : 17702 (lingeling) R 17698 17698 17121 0 -1 4202496 48195 0 0 0 118222 6 0 0 25 0 1 0 519455650 34484224 7613 33554432000 4194304 4352236 140736874418224 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17702/statm: 8419 7613 127 39 0 7499 0
[pid=17703] ppid=17698 vsize=98348 CPUtime=587.5 cores=7
/proc/17703/stat : 17703 (march_hi) R 17698 17698 17121 0 -1 4202496 18202 0 0 0 58741 9 0 0 25 0 1 0 519455650 100708352 16978 33554432000 4194304 4331988 140735605524912 18446744073709551615 4287549 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17703/statm: 24587 16978 175 34 0 23029 0
Current children cumulated CPU time (s) 4721.98
Current children cumulated vsize (KiB) 740576



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 10.20 10.12 9.52 6/181 17740
/proc/meminfo: memFree=17687528/32951124 swapFree=67111528/67111528
[pid=17698] ppid=17696 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/17698/stat : 17698 (ppfolio) S 17696 17698 17121 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 519455650 18763776 390 33554432000 4194304 4287625 140736693474576 18446744073709551615 216634535439 0 0 4096 65536 4605300334066139135 0 0 17 1 0 0 0
/proc/17698/statm: 4581 390 341 23 0 91 0
[pid=17699] ppid=17698 vsize=85536 CPUtime=1201.91 cores=1
/proc/17699/stat : 17699 (cryptominisat) R 17698 17698 17121 0 -1 4202496 161793 0 0 0 120129 62 0 0 25 0 1 0 519455650 87588864 17574 33554432000 4194304 4587354 140736003094400 18446744073709551615 4238568 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/17699/statm: 21384 17574 336 96 0 17946 0
[pid=17700] ppid=17698 vsize=47924 CPUtime=1201.12 cores=5
/proc/17700/stat : 17700 (clasp) R 17698 17698 17121 0 -1 4202496 9009 0 0 0 120111 1 0 0 25 0 1 0 519455650 49074176 8482 33554432000 4194304 4785740 140737324327088 18446744073709551615 4287143 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/17700/statm: 11981 8482 341 145 0 9026 0
[pid=17701] ppid=17698 vsize=456768 CPUtime=598.12 cores=7
/proc/17701/stat : 17701 (TNM) R 17698 17698 17121 0 -1 4202496 597 0 0 0 59812 0 0 0 25 0 1 0 519455650 467730432 493 33554432000 134512640 134534564 4291879840 18446744073709551615 134528511 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17701/statm: 114192 493 95 6 0 112858 0
[pid=17702] ppid=17698 vsize=31884 CPUtime=1201.88 cores=3
/proc/17702/stat : 17702 (lingeling) R 17698 17698 17121 0 -1 4202496 49027 0 0 0 120182 6 0 0 25 0 1 0 519455650 32649216 7165 33554432000 4194304 4352236 140736874418224 18446744073709551615 4328095 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/17702/statm: 7971 7165 127 39 0 7051 0
[pid=17703] ppid=17698 vsize=98348 CPUtime=597.24 cores=7
/proc/17703/stat : 17703 (march_hi) R 17698 17698 17121 0 -1 4202496 18473 0 0 0 59715 9 0 0 25 0 1 0 519455650 100708352 17249 33554432000 4194304 4331988 140735605524912 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/17703/statm: 24587 17249 175 34 0 23029 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 738784

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 17698 and gives
#  childrusage.ru_utime.tv_sec=2402
#  childrusage.ru_utime.tv_usec=944696
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=98984
# CPU time returned by wait4() is 2403.04
# while last known CPU time is 4800.27
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.91
CPU time (s): 4800.27
CPU user time (s): 4799.49
CPU system time (s): 0.78
CPU usage (%): 399.386
Max. virtual memory (cumulated for all children) (KiB): 1483792

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2402.94
system time used= 0.098984
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 58576
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 545
involuntary context switches= 6103

runsolver used 1.9407 second user time and 4.6223 second system time

The end

Launcher Data 

Begin job on node103 at 2011-04-29 14:02:10
IDJOB=3191323
IDBENCH=83054
IDSOLVER=1612
FILE ID=node103/3191323-1304078530
RUNJOBID= node103-1304078530-17674
PBS_JOBID= 13170491
Free space on /tmp= 71508 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/medium/unif-k7-r89-v90-c8010-S1897189325-062.UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3191323-1304078530/watcher-3191323-1304078530 -o /tmp/evaluation-result-3191323-1304078530/solver-3191323-1304078530 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3191323-1304078530.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 28731ba8f9c5274ff0cd070fd7aeff45
RANDOM SEED=888033288

node103.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17888408 kB
Buffers:       2147860 kB
Cached:       12264768 kB
SwapCached:          0 kB
Active:        5594184 kB
Inactive:      8885480 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17888408 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           20336 kB
Writeback:           0 kB
AnonPages:       66896 kB
Mapped:          15644 kB
Slab:           520040 kB
PageTables:       5264 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   668360 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71500 MiB
End job on node103 at 2011-04-29 14:22:14