Trace number 3289904

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.24 1201.71

General information on the benchmark

Nameapplication/kullmann/AES/Benchmarks/
128/aes_128_2_keyfind_1.cnf
MD5SUM19e0b91e7d093f7cb35b11b46708a49b
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2000
Number of clauses20544
Sum of the clauses size111944
Maximum clause length8
Minimum clause length1
Number of clauses of size 1272
Number of clauses of size 2256
Number of clauses of size 33520
Number of clauses of size 4640
Number of clauses of size 54096
Number of clauses of size over 511760

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 4
0.00/0.00	Starting TNM 2009 on core(s) 6
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 6

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3289904-1304087809/watcher-3289904-1304087809 -o /tmp/evaluation-result-3289904-1304087809/solver-3289904-1304087809 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3289904-1304087809.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.84 7.81 7.93 2/172 14183
/proc/meminfo: memFree=23109196/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) R 14181 14183 14150 0 -1 4202496 518 0 0 0 0 0 0 0 22 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0

[startup+0.221766 s]
/proc/loadavg: 6.84 7.81 7.93 2/172 14183
/proc/meminfo: memFree=23109196/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) R 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 21 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.321746 s]
/proc/loadavg: 6.84 7.81 7.93 2/172 14183
/proc/meminfo: memFree=23109196/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) R 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 21 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.400732 s]
/proc/loadavg: 6.84 7.81 7.93 2/172 14183
/proc/meminfo: memFree=23109196/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 20 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700679 s]
/proc/loadavg: 6.84 7.81 7.93 2/172 14183
/proc/meminfo: memFree=23109196/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 17 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50053 s]
/proc/loadavg: 6.84 7.81 7.93 11/188 14199
/proc/meminfo: memFree=23016672/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=400628 CPUtime=1.49 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 5711 0 0 0 148 1 0 0 23 0 1 0 520381400 410243072 3271 33554432000 4194304 4587354 140735373572480 18446744073709551615 4238499 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 100157 3272 338 96 0 96719 0
[pid=14185] ppid=14183 vsize=19576 CPUtime=1.49 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 2446 0 0 0 149 0 0 0 25 0 1 0 520381400 20045824 2243 33554432000 4194304 4785740 140737320941152 18446744073709551615 4287190 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 4894 2243 341 145 0 1939 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=0.78 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 78 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134530504 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=8204 CPUtime=1.49 cores=2
/proc/14187/stat : 14187 (lingeling) R 14183 14183 14150 0 -1 4202496 1687 0 0 0 149 0 0 0 25 0 1 0 520381400 8400896 1249 33554432000 4194304 4352236 140736030294816 18446744073709551615 4328016 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14187/statm: 2051 1249 127 39 0 1131 0
[pid=14188] ppid=14183 vsize=24456 CPUtime=0.69 cores=6
/proc/14188/stat : 14188 (march_hi) R 14183 14183 14150 0 -1 4202496 10252 0 0 0 69 0 0 0 25 0 1 0 520381400 25042944 3534 33554432000 4194304 4331988 140737232104016 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14188/statm: 6114 3534 176 34 0 4556 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 929128

[startup+3.10123 s]
/proc/loadavg: 7.09 7.85 7.94 11/188 14199
/proc/meminfo: memFree=23007372/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=38152 CPUtime=3.09 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 8688 0 0 0 307 2 0 0 25 0 1 0 520381400 39067648 4061 33554432000 4194304 4587354 140735373572480 18446744073709551615 4378507 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 9538 4061 338 96 0 6100 0
[pid=14185] ppid=14183 vsize=21032 CPUtime=3.09 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 2799 0 0 0 309 0 0 0 25 0 1 0 520381400 21536768 2596 33554432000 4194304 4785740 140737320941152 18446744073709551615 4553046 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 5258 2596 341 145 0 2303 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=1.57 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 157 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=8984 CPUtime=3.09 cores=2
/proc/14187/stat : 14187 (lingeling) R 14183 14183 14150 0 -1 4202496 2458 0 0 0 309 0 0 0 25 0 1 0 520381400 9199616 1444 33554432000 4194304 4352236 140736030294816 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14187/statm: 2246 1444 127 39 0 1326 0
[pid=14188] ppid=14183 vsize=24456 CPUtime=1.49 cores=6
/proc/14188/stat : 14188 (march_hi) R 14183 14183 14150 0 -1 4202496 10281 0 0 0 149 0 0 0 25 0 1 0 520381400 25042944 3563 33554432000 4194304 4331988 140737232104016 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14188/statm: 6114 3563 176 34 0 4556 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 568888

[startup+6.30063 s]
/proc/loadavg: 7.09 7.85 7.94 11/188 14199
/proc/meminfo: memFree=22983028/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=407500 CPUtime=6.29 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 12711 0 0 0 626 3 0 0 25 0 1 0 520381400 417280000 5422 33554432000 4194304 4587354 140735373572480 18446744073709551615 4238591 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 101875 5422 338 96 0 98437 0
[pid=14185] ppid=14183 vsize=23340 CPUtime=6.29 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 3353 0 0 0 629 0 0 0 25 0 1 0 520381400 23900160 3150 33554432000 4194304 4785740 140737320941152 18446744073709551615 4278070 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 5835 3150 341 145 0 2880 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=3.16 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 316 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134529838 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=11036 CPUtime=6.28 cores=2
/proc/14187/stat : 14187 (lingeling) R 14183 14183 14150 0 -1 4202496 3611 0 0 0 628 0 0 0 25 0 1 0 520381400 11300864 1957 33554432000 4194304 4352236 140736030294816 18446744073709551615 4327936 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14187/statm: 2759 1957 127 39 0 1839 0
[pid=14188] ppid=14183 vsize=24456 CPUtime=3.09 cores=6
/proc/14188/stat : 14188 (march_hi) R 14183 14183 14150 0 -1 4202496 10294 0 0 0 309 0 0 0 25 0 1 0 520381400 25042944 3576 33554432000 4194304 4331988 140737232104016 18446744073709551615 4242612 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14188/statm: 6114 3576 176 34 0 4556 0
Current children cumulated CPU time (s) 25.11
Current children cumulated vsize (KiB) 942596

[startup+12.7004 s]
/proc/loadavg: 7.62 7.93 7.97 11/188 14199
/proc/meminfo: memFree=22961244/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=411580 CPUtime=12.69 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 16517 0 0 0 1265 4 0 0 25 0 1 0 520381400 421457920 6672 33554432000 4194304 4587354 140735373572480 18446744073709551615 4239534 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 102895 6672 338 96 0 99457 0
[pid=14185] ppid=14183 vsize=28172 CPUtime=12.68 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 4547 0 0 0 1268 0 0 0 25 0 1 0 520381400 28848128 4344 33554432000 4194304 4785740 140737320941152 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 7043 4344 341 145 0 4088 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=6.32 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 632 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134528295 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=12076 CPUtime=12.68 cores=2
/proc/14187/stat : 14187 (lingeling) R 14183 14183 14150 0 -1 4202496 4447 0 0 0 1268 0 0 0 25 0 1 0 520381400 12365824 2217 33554432000 4194304 4352236 140736030294816 18446744073709551615 4328126 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14187/statm: 3019 2217 127 39 0 2099 0
[pid=14188] ppid=14183 vsize=24696 CPUtime=6.29 cores=6
/proc/14188/stat : 14188 (march_hi) R 14183 14183 14150 0 -1 4202496 10349 0 0 0 629 0 0 0 25 0 1 0 520381400 25288704 3631 33554432000 4194304 4331988 140737232104016 18446744073709551615 4242575 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14188/statm: 6174 3631 176 34 0 4616 0
Current children cumulated CPU time (s) 50.66
Current children cumulated vsize (KiB) 952788

[startup+25.507 s]
/proc/loadavg: 7.98 8.00 7.99 11/188 14200
/proc/meminfo: memFree=22928788/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=41572 CPUtime=25.5 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 30048 0 0 0 2544 6 0 0 25 0 1 0 520381400 42569728 6991 33554432000 4194304 4587354 140735373572480 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 10393 6991 338 96 0 6955 0
[pid=14185] ppid=14183 vsize=31744 CPUtime=25.49 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 5383 0 0 0 2549 0 0 0 25 0 1 0 520381400 32505856 5180 33554432000 4194304 4785740 140737320941152 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 7936 5180 341 145 0 4981 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=12.7 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 1270 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134528523 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=17076 CPUtime=25.49 cores=2
/proc/14187/stat : 14187 (lingeling) R 14183 14183 14150 0 -1 4202496 8193 0 0 0 2548 1 0 0 25 0 1 0 520381400 17485824 3467 33554432000 4194304 4352236 140736030294816 18446744073709551615 4328059 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14187/statm: 4269 3467 127 39 0 3349 0
[pid=14188] ppid=14183 vsize=25068 CPUtime=12.64 cores=6
/proc/14188/stat : 14188 (march_hi) R 14183 14183 14150 0 -1 4202496 10393 0 0 0 1264 0 0 0 25 0 1 0 520381400 25669632 3675 33554432000 4194304 4331988 140737232104016 18446744073709551615 4242607 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14188/statm: 6267 3675 176 34 0 4709 0
Current children cumulated CPU time (s) 101.82
Current children cumulated vsize (KiB) 591724

[startup+51.1013 s]
/proc/loadavg: 8.73 8.18 8.05 12/188 14201
/proc/meminfo: memFree=22902468/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=423640 CPUtime=51.09 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 42086 0 0 0 5100 9 0 0 25 0 1 0 520381400 433807360 8490 33554432000 4194304 4587354 140735373572480 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 105910 8490 338 96 0 102472 0
[pid=14185] ppid=14183 vsize=36368 CPUtime=51.08 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 6488 0 0 0 5108 0 0 0 25 0 1 0 520381400 37240832 6285 33554432000 4194304 4785740 140737320941152 18446744073709551615 4553440 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 9092 6285 341 145 0 6137 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=25.42 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 2542 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134528659 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=17460 CPUtime=51.08 cores=2

################
# More data... #
################

/proc/14188/statm: 11294 7335 176 34 0 9736 0
Current children cumulated CPU time (s) 3764.02
Current children cumulated vsize (KiB) 1084404

[startup+1002.31 s]
/proc/loadavg: 10.12 10.09 9.40 11/188 14226
/proc/meminfo: memFree=22642556/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=81828 CPUtime=1002.32 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 248219 0 0 0 100159 73 0 0 25 0 1 0 520381400 83791872 16856 33554432000 4194304 4587354 140735373572480 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 20457 16856 338 96 0 17019 0
[pid=14185] ppid=14183 vsize=91076 CPUtime=1002.23 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 19328 0 0 0 100220 3 0 0 25 0 1 0 520381400 93261824 19053 33554432000 4194304 4785740 140737320941152 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 22769 19053 341 145 0 19814 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=498.43 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 49843 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134529765 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=22204 CPUtime=1002.32 cores=2
/proc/14187/stat : 14187 (lingeling) R 14183 14183 14150 0 -1 4202496 60739 0 0 0 100223 9 0 0 25 0 1 0 520381400 22736896 4749 33554432000 4194304 4352236 140736030294816 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14187/statm: 5551 4749 127 39 0 4631 0
[pid=14188] ppid=14183 vsize=45176 CPUtime=498.41 cores=6
/proc/14188/stat : 14188 (march_hi) R 14183 14183 14150 0 -1 4202496 14260 0 0 0 49839 2 0 0 25 0 1 0 520381400 46260224 7542 33554432000 4194304 4331988 140737232104016 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14188/statm: 11294 7542 176 34 0 9736 0
Current children cumulated CPU time (s) 4003.71
Current children cumulated vsize (KiB) 716548

[startup+1062.3 s]
/proc/loadavg: 10.15 10.10 9.44 11/188 14227
/proc/meminfo: memFree=22599756/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=489744 CPUtime=1062.32 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 254287 0 0 0 106157 75 0 0 25 0 1 0 520381400 501497856 22026 33554432000 4194304 4587354 140735373572480 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 122436 22026 338 96 0 118998 0
[pid=14185] ppid=14183 vsize=92008 CPUtime=1062.22 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 19580 0 0 0 106219 3 0 0 25 0 1 0 520381400 94216192 19305 33554432000 4194304 4785740 140737320941152 18446744073709551615 4550319 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 23002 19305 341 145 0 20047 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=528.31 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 52831 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134528661 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=31676 CPUtime=1062.32 cores=2
/proc/14187/stat : 14187 (lingeling) R 14183 14183 14150 0 -1 4202496 64515 0 0 0 106222 10 0 0 25 0 1 0 520381400 32436224 7117 33554432000 4194304 4352236 140736030294816 18446744073709551615 4329086 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14187/statm: 7919 7117 127 39 0 6999 0
[pid=14188] ppid=14183 vsize=45176 CPUtime=528.21 cores=6
/proc/14188/stat : 14188 (march_hi) R 14183 14183 14150 0 -1 4202496 14486 0 0 0 52819 2 0 0 25 0 1 0 520381400 46260224 7768 33554432000 4194304 4331988 140737232104016 18446744073709551615 4299299 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14188/statm: 11294 7768 176 34 0 9736 0
Current children cumulated CPU time (s) 4243.38
Current children cumulated vsize (KiB) 1134868

[startup+1122.3 s]
/proc/loadavg: 10.14 10.11 9.48 11/188 14229
/proc/meminfo: memFree=22603192/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=471972 CPUtime=1122.32 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 261876 0 0 0 112154 78 0 0 25 0 1 0 520381400 483299328 22997 33554432000 4194304 4587354 140735373572480 18446744073709551615 4238755 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 117993 22997 338 96 0 114555 0
[pid=14185] ppid=14183 vsize=93364 CPUtime=1122.22 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 19841 0 0 0 112219 3 0 0 25 0 1 0 520381400 95604736 19566 33554432000 4194304 4785740 140737320941152 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 23341 19566 341 145 0 20386 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=558.12 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 55812 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134530717 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=23740 CPUtime=1122.32 cores=2
/proc/14187/stat : 14187 (lingeling) R 14183 14183 14150 0 -1 4202496 64579 0 0 0 112222 10 0 0 25 0 1 0 520381400 24309760 5133 33554432000 4194304 4352236 140736030294816 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14187/statm: 5935 5133 127 39 0 5015 0
[pid=14188] ppid=14183 vsize=45176 CPUtime=558.08 cores=6
/proc/14188/stat : 14188 (march_hi) R 14183 14183 14150 0 -1 4202496 14748 0 0 0 55805 3 0 0 25 0 1 0 520381400 46260224 8030 33554432000 4194304 4331988 140737232104016 18446744073709551615 4242612 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14188/statm: 11294 8030 176 34 0 9736 0
Current children cumulated CPU time (s) 4483.06
Current children cumulated vsize (KiB) 1110516

[startup+1182.3 s]
/proc/loadavg: 10.14 10.12 9.52 11/188 14231
/proc/meminfo: memFree=22594488/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=480408 CPUtime=1182.32 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 269326 0 0 0 118151 81 0 0 25 0 1 0 520381400 491937792 23602 33554432000 4194304 4587354 140735373572480 18446744073709551615 244446641812 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 120102 23602 338 96 0 116664 0
[pid=14185] ppid=14183 vsize=93968 CPUtime=1182.21 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 20014 0 0 0 118218 3 0 0 25 0 1 0 520381400 96223232 19739 33554432000 4194304 4785740 140737320941152 18446744073709551615 4416768 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 23492 19739 341 145 0 20537 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=587.97 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 58797 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134530332 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=30524 CPUtime=1182.32 cores=2
/proc/14187/stat : 14187 (lingeling) R 14183 14183 14150 0 -1 4202496 67459 0 0 0 118222 10 0 0 25 0 1 0 520381400 31256576 6829 33554432000 4194304 4352236 140736030294816 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14187/statm: 7631 6829 127 39 0 6711 0
[pid=14188] ppid=14183 vsize=45176 CPUtime=587.9 cores=6
/proc/14188/stat : 14188 (march_hi) R 14183 14183 14150 0 -1 4202496 14975 0 0 0 58787 3 0 0 25 0 1 0 520381400 46260224 8257 33554432000 4194304 4331988 140737232104016 18446744073709551615 4228546 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14188/statm: 11294 8257 176 34 0 9736 0
Current children cumulated CPU time (s) 4722.72
Current children cumulated vsize (KiB) 1126340



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 10.17 10.13 9.54 11/188 14231
/proc/meminfo: memFree=22591628/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=465936 CPUtime=1201.73 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 273807 0 0 0 120090 83 0 0 25 0 1 0 520381400 477118464 21259 33554432000 4194304 4587354 140735373572480 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 116484 21259 338 96 0 113046 0
[pid=14185] ppid=14183 vsize=94408 CPUtime=1201.62 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 20088 0 0 0 120159 3 0 0 25 0 1 0 520381400 96673792 19813 33554432000 4194304 4785740 140737320941152 18446744073709551615 4552547 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 23602 19813 341 145 0 20647 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=597.61 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 59761 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134530549 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=30652 CPUtime=1201.73 cores=2
/proc/14187/stat : 14187 (lingeling) R 14183 14183 14150 0 -1 4202496 67491 0 0 0 120163 10 0 0 25 0 1 0 520381400 31387648 6861 33554432000 4194304 4352236 140736030294816 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14187/statm: 7663 6861 127 39 0 6743 0
[pid=14188] ppid=14183 vsize=47816 CPUtime=597.55 cores=6
/proc/14188/stat : 14188 (march_hi) R 14183 14183 14150 0 -1 4202496 15052 0 0 0 59752 3 0 0 25 0 1 0 520381400 48963584 8334 33554432000 4194304 4331988 140737232104016 18446744073709551615 4287105 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14188/statm: 11954 8334 176 34 0 10396 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 1115076

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 10.17 10.13 9.54 11/188 14231
/proc/meminfo: memFree=22591628/32951124 swapFree=67111524/67111528
[pid=14183] ppid=14181 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/14183/stat : 14183 (ppfolio) S 14181 14183 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 520381399 18763776 389 33554432000 4194304 4287625 140734902749952 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 4 0 0 0
/proc/14183/statm: 4581 389 341 23 0 91 0
[pid=14184] ppid=14183 vsize=465936 CPUtime=1201.73 cores=0
/proc/14184/stat : 14184 (cryptominisat) R 14183 14183 14150 0 -1 4202496 273807 0 0 0 120090 83 0 0 25 0 1 0 520381400 477118464 21259 33554432000 4194304 4587354 140735373572480 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/14184/statm: 116484 21259 338 96 0 113046 0
[pid=14185] ppid=14183 vsize=94408 CPUtime=1201.62 cores=4
/proc/14185/stat : 14185 (clasp) R 14183 14183 14150 0 -1 4202496 20088 0 0 0 120159 3 0 0 25 0 1 0 520381400 96673792 19813 33554432000 4194304 4785740 140737320941152 18446744073709551615 4552547 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/14185/statm: 23602 19813 341 145 0 20647 0
[pid=14186] ppid=14183 vsize=457940 CPUtime=597.61 cores=6
/proc/14186/stat : 14186 (TNM) R 14183 14183 14150 0 -1 4202496 1055 0 0 0 59761 0 0 0 25 0 1 0 520381400 468930560 952 33554432000 134512640 134534564 4288191824 18446744073709551615 134530549 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14186/statm: 114485 952 95 6 0 113151 0
[pid=14187] ppid=14183 vsize=30652 CPUtime=1201.73 cores=2
/proc/14187/stat : 14187 (lingeling) R 14183 14183 14150 0 -1 4202496 67491 0 0 0 120163 10 0 0 25 0 1 0 520381400 31387648 6861 33554432000 4194304 4352236 140736030294816 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/14187/statm: 7663 6861 127 39 0 6743 0
[pid=14188] ppid=14183 vsize=47816 CPUtime=597.55 cores=6
/proc/14188/stat : 14188 (march_hi) R 14183 14183 14150 0 -1 4202496 15052 0 0 0 59752 3 0 0 25 0 1 0 520381400 48963584 8334 33554432000 4194304 4331988 140737232104016 18446744073709551615 4287105 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/14188/statm: 11954 8334 176 34 0 10396 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 1115076

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 14183 and gives
#  childrusage.ru_utime.tv_sec=1799
#  childrusage.ru_utime.tv_usec=251472
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=120981
# CPU time returned by wait4() is 1799.37
# while last known CPU time is 4800.24
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.24
CPU user time (s): 4799.25
CPU system time (s): 0.99
CPU usage (%): 399.45
Max. virtual memory (cumulated for all children) (KiB): 1510448

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1799.25
system time used= 0.120981
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 69068
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 452
involuntary context switches= 14421

runsolver used 1.79273 second user time and 4.77127 second system time

The end

Launcher Data 

Begin job on node105 at 2011-04-29 16:36:50
IDJOB=3289904
IDBENCH=83410
IDSOLVER=1612
FILE ID=node105/3289904-1304087809
RUNJOBID= node105-1304087809-14168
PBS_JOBID= 13170561
Free space on /tmp= 71940 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/kullmann/AES/Benchmarks/128/aes_128_2_keyfind_1.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3289904-1304087809/watcher-3289904-1304087809 -o /tmp/evaluation-result-3289904-1304087809/solver-3289904-1304087809 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3289904-1304087809.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 19e0b91e7d093f7cb35b11b46708a49b
RANDOM SEED=683605178

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23109468 kB
Buffers:       2154764 kB
Cached:        7075136 kB
SwapCached:          4 kB
Active:         523944 kB
Inactive:      8768724 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23109468 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           21504 kB
Writeback:           0 kB
AnonPages:       63320 kB
Mapped:          13944 kB
Slab:           484036 kB
PageTables:       5028 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   183448 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71920 MiB
End job on node105 at 2011-04-29 16:56:53