Trace number 3190814

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.17 1201.71

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v30000-c126000-S629207485-001-UNKNOWN.cnf
MD5SUM33c09da85657b6bbed23989098d6baf7
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark50.0704
Satisfiable
(Un)Satisfiability was proved
Number of variables30000
Number of clauses126000
Sum of the clauses size378000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3126000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 4
0.00/0.00	Starting TNM 2009 on core(s) 6
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 6

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3190814-1304036606/watcher-3190814-1304036606 -o /tmp/evaluation-result-3190814-1304036606/solver-3190814-1304036606 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3190814-1304036606.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.78 10.14 10.08 6/186 27452
/proc/meminfo: memFree=31025588/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) R 27450 27452 26347 0 -1 4202496 518 0 0 0 0 0 0 0 25 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0

[startup+0.0470571 s]
/proc/loadavg: 9.78 10.14 10.08 6/186 27452
/proc/meminfo: memFree=31025588/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) R 27450 27452 26347 0 -1 4202496 518 0 0 0 0 0 0 0 25 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.101047 s]
/proc/loadavg: 9.78 10.14 10.08 6/186 27452
/proc/meminfo: memFree=31025588/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) R 27450 27452 26347 0 -1 4202496 518 0 0 0 0 0 0 0 25 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.301012 s]
/proc/loadavg: 9.78 10.14 10.08 6/186 27452
/proc/meminfo: memFree=31025588/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 23 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700938 s]
/proc/loadavg: 9.78 10.14 10.08 6/186 27452
/proc/meminfo: memFree=31025588/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 20 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50079 s]
/proc/loadavg: 9.78 10.14 10.08 12/192 27458
/proc/meminfo: memFree=30931796/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 20 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=51188 CPUtime=1.48 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 10902 0 0 0 147 1 0 0 25 0 1 0 515258472 52416512 8488 33554432000 4194304 4587354 140736096288528 18446744073709551615 4238297 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 12797 8488 326 96 0 9359 0
[pid=27454] ppid=27452 vsize=28552 CPUtime=1.48 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 5880 0 0 0 148 0 0 0 25 0 1 0 515258472 29237248 4447 33554432000 4194304 4785740 140734352268960 18446744073709551615 4553032 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 7138 4447 340 145 0 4183 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=0.74 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 74 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134529640 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=16184 CPUtime=1.48 cores=2
/proc/27456/stat : 27456 (lingeling) R 27452 27452 26347 0 -1 4202496 5274 0 0 0 148 0 0 0 25 0 1 0 515258472 16572416 3148 33554432000 4194304 4352236 140734989521088 18446744073709551615 4328047 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/27456/statm: 4046 3148 127 39 0 3126 0
[pid=27457] ppid=27452 vsize=24268 CPUtime=0.73 cores=6
/proc/27457/stat : 27457 (march_hi) R 27452 27452 26347 0 -1 4202496 17750 0 0 0 71 2 0 0 25 0 1 0 515258472 24850432 3411 33554432000 4194304 4331988 140736448593520 18446744073709551615 4220842 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27457/statm: 6067 3411 141 34 0 4509 0
Current children cumulated CPU time (s) 5.91
Current children cumulated vsize (KiB) 602540

[startup+3.1005 s]
/proc/loadavg: 9.78 10.14 10.08 11/192 27458
/proc/meminfo: memFree=30923364/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 18 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=88692 CPUtime=3.09 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 11965 0 0 0 307 2 0 0 25 0 1 0 515258472 90820608 9544 33554432000 4194304 4587354 140736096288528 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 22173 9544 326 96 0 18735 0
[pid=27454] ppid=27452 vsize=34136 CPUtime=3.08 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 7262 0 0 0 308 0 0 0 25 0 1 0 515258472 34955264 5829 33554432000 4194304 4785740 140734352268960 18446744073709551615 4279296 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 8534 5829 340 145 0 5579 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=1.54 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 154 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134530614 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=16184 CPUtime=3.08 cores=2
/proc/27456/stat : 27456 (lingeling) R 27452 27452 26347 0 -1 4202496 5274 0 0 0 308 0 0 0 25 0 1 0 515258472 16572416 3148 33554432000 4194304 4352236 140734989521088 18446744073709551615 4328459 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/27456/statm: 4046 3148 127 39 0 3126 0
[pid=27457] ppid=27452 vsize=65268 CPUtime=1.52 cores=6
/proc/27457/stat : 27457 (march_hi) R 27452 27452 26347 0 -1 4202496 35250 0 0 0 148 4 0 0 25 0 1 0 515258472 66834432 7398 33554432000 4194304 4331988 140736448593520 18446744073709551615 4295989 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27457/statm: 16317 7398 151 34 0 14759 0
Current children cumulated CPU time (s) 12.31
Current children cumulated vsize (KiB) 686628

[startup+6.30092 s]
/proc/loadavg: 10.04 10.19 10.10 11/192 27458
/proc/meminfo: memFree=30863352/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=88692 CPUtime=6.28 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 13803 0 0 0 626 2 0 0 25 0 1 0 515258472 90820608 11376 33554432000 4194304 4587354 140736096288528 18446744073709551615 4250248 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 22173 11376 326 96 0 18735 0
[pid=27454] ppid=27452 vsize=46456 CPUtime=6.28 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 10342 0 0 0 627 1 0 0 25 0 1 0 515258472 47570944 8909 33554432000 4194304 4785740 140734352268960 18446744073709551615 4553068 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 11614 8909 340 145 0 8659 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=3.14 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 314 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134530476 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=16184 CPUtime=6.28 cores=2
/proc/27456/stat : 27456 (lingeling) R 27452 27452 26347 0 -1 4202496 5274 0 0 0 628 0 0 0 25 0 1 0 515258472 16572416 3148 33554432000 4194304 4352236 140734989521088 18446744073709551615 4328032 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/27456/statm: 4046 3148 127 39 0 3126 0
[pid=27457] ppid=27452 vsize=67304 CPUtime=3.1 cores=6
/proc/27457/stat : 27457 (march_hi) R 27452 27452 26347 0 -1 4202496 41679 0 0 0 305 5 0 0 25 0 1 0 515258472 68919296 13827 33554432000 4194304 4331988 140736448593520 18446744073709551615 4229078 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27457/statm: 16826 13827 173 34 0 15268 0
Current children cumulated CPU time (s) 25.08
Current children cumulated vsize (KiB) 700984

[startup+12.7008 s]
/proc/loadavg: 10.03 10.18 10.10 11/192 27458
/proc/meminfo: memFree=30835696/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=141016 CPUtime=12.68 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 19087 0 0 0 1264 4 0 0 25 0 1 0 515258472 144400384 10523 33554432000 4194304 4587354 140736096288528 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 35254 10523 331 96 0 31816 0
[pid=27454] ppid=27452 vsize=71208 CPUtime=12.69 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 16476 0 0 0 1266 3 0 0 25 0 1 0 515258472 72916992 15043 33554432000 4194304 4785740 140734352268960 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 17802 15043 340 145 0 14847 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=6.31 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 631 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134528659 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=21520 CPUtime=12.68 cores=2
/proc/27456/stat : 27456 (lingeling) R 27452 27452 26347 0 -1 4202496 6672 0 0 0 1268 0 0 0 25 0 1 0 515258472 22036480 4546 33554432000 4194304 4352236 140734989521088 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/27456/statm: 5380 4546 127 39 0 4460 0
[pid=27457] ppid=27452 vsize=71580 CPUtime=6.3 cores=6
/proc/27457/stat : 27457 (march_hi) R 27452 27452 26347 0 -1 4202496 42594 0 0 0 625 5 0 0 25 0 1 0 515258472 73297920 14742 33554432000 4194304 4331988 140736448593520 18446744073709551615 4211509 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27457/statm: 17895 14742 173 34 0 16337 0
Current children cumulated CPU time (s) 50.66
Current children cumulated vsize (KiB) 787672

[startup+25.5004 s]
/proc/loadavg: 10.03 10.17 10.10 12/192 27459
/proc/meminfo: memFree=30739820/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=106492 CPUtime=25.48 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 56151 0 0 0 2537 11 0 0 25 0 1 0 515258472 109047808 21758 33554432000 4194304 4587354 140736096288528 18446744073709551615 4388350 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 26623 21758 337 96 0 23185 0
[pid=27454] ppid=27452 vsize=101812 CPUtime=25.49 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 24091 0 0 0 2544 5 0 0 25 0 1 0 515258472 104255488 22658 33554432000 4194304 4785740 140734352268960 18446744073709551615 4553068 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 25453 22658 340 145 0 22498 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=12.64 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 1264 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134528330 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=24020 CPUtime=25.48 cores=2
/proc/27456/stat : 27456 (lingeling) R 27452 27452 26347 0 -1 4202496 7297 0 0 0 2548 0 0 0 25 0 1 0 515258472 24596480 5171 33554432000 4194304 4352236 140734989521088 18446744073709551615 4329373 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/27456/statm: 6005 5171 127 39 0 5085 0
[pid=27457] ppid=27452 vsize=75016 CPUtime=12.69 cores=6
/proc/27457/stat : 27457 (march_hi) R 27452 27452 26347 0 -1 4202496 43553 0 0 0 1264 5 0 0 25 0 1 0 515258472 76816384 15701 33554432000 4194304 4331988 140736448593520 18446744073709551615 4288317 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27457/statm: 18754 15701 173 34 0 17196 0
Current children cumulated CPU time (s) 101.78
Current children cumulated vsize (KiB) 789688

[startup+51.1008 s]
/proc/loadavg: 10.02 10.16 10.09 11/192 27459
/proc/meminfo: memFree=30740328/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=62256 CPUtime=51.09 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 83510 0 0 0 5089 20 0 0 25 0 1 0 515258472 63750144 11849 33554432000 4194304 4587354 140736096288528 18446744073709551615 4238252 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 15564 11849 337 96 0 12126 0
[pid=27454] ppid=27452 vsize=126544 CPUtime=51.09 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 30285 0 0 0 5103 6 0 0 25 0 1 0 515258472 129581056 28852 33554432000 4194304 4785740 140734352268960 18446744073709551615 4287050 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 31636 28852 340 145 0 28681 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=25.4 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 2540 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=26976 CPUtime=51.08 cores=2

################
# More data... #
################

/proc/27457/statm: 20140 17080 173 34 0 18582 0
Current children cumulated CPU time (s) 3763.98
Current children cumulated vsize (KiB) 1193308

[startup+1002.3 s]
/proc/loadavg: 10.42 10.35 10.22 11/192 27518
/proc/meminfo: memFree=30657576/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=201052 CPUtime=1002.29 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 622202 0 0 0 100011 218 0 0 25 0 1 0 515258472 205877248 25917 33554432000 4194304 4587354 140736096288528 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 50263 25917 337 96 0 46825 0
[pid=27454] ppid=27452 vsize=371928 CPUtime=1002.31 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 90337 0 0 0 100207 24 0 0 25 0 1 0 515258472 380854272 88904 33554432000 4194304 4785740 140734352268960 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 92982 88904 340 145 0 90027 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=498.41 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 49841 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134530158 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=26976 CPUtime=1002.21 cores=2
/proc/27456/stat : 27456 (lingeling) R 27452 27452 26347 0 -1 4202496 8036 0 0 0 100220 1 0 0 25 0 1 0 515258472 27623424 5910 33554432000 4194304 4352236 140734989521088 18446744073709551615 4328682 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/27456/statm: 6744 5910 127 39 0 5824 0
[pid=27457] ppid=27452 vsize=80560 CPUtime=498.42 cores=6
/proc/27457/stat : 27457 (march_hi) R 27452 27452 26347 0 -1 4202496 44951 0 0 0 49836 6 0 0 25 0 1 0 515258472 82493440 17099 33554432000 4194304 4331988 140736448593520 18446744073709551615 4263253 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27457/statm: 20140 17099 173 34 0 18582 0
Current children cumulated CPU time (s) 4003.64
Current children cumulated vsize (KiB) 1162864

[startup+1062.3 s]
/proc/loadavg: 10.24 10.31 10.21 11/192 27519
/proc/meminfo: memFree=30622132/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=197020 CPUtime=1062.29 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 642173 0 0 0 106000 229 0 0 25 0 1 0 515258472 201748480 33650 33554432000 4194304 4587354 140736096288528 18446744073709551615 4238299 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 49255 33650 337 96 0 45817 0
[pid=27454] ppid=27452 vsize=391468 CPUtime=1062.32 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 95207 0 0 0 106206 26 0 0 25 0 1 0 515258472 400863232 93774 33554432000 4194304 4785740 140734352268960 18446744073709551615 4286973 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 97867 93774 340 145 0 94912 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=528.26 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 52826 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134530379 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=26976 CPUtime=1062.21 cores=2
/proc/27456/stat : 27456 (lingeling) R 27452 27452 26347 0 -1 4202496 8036 0 0 0 106220 1 0 0 25 0 1 0 515258472 27623424 5910 33554432000 4194304 4352236 140734989521088 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/27456/statm: 6744 5910 127 39 0 5824 0
[pid=27457] ppid=27452 vsize=80692 CPUtime=528.25 cores=6
/proc/27457/stat : 27457 (march_hi) R 27452 27452 26347 0 -1 4202496 44975 0 0 0 52819 6 0 0 25 0 1 0 515258472 82628608 17123 33554432000 4194304 4331988 140736448593520 18446744073709551615 4265440 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27457/statm: 20173 17123 173 34 0 18615 0
Current children cumulated CPU time (s) 4243.33
Current children cumulated vsize (KiB) 1178504

[startup+1122.3 s]
/proc/loadavg: 10.26 10.31 10.21 11/192 27521
/proc/meminfo: memFree=30616016/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=117848 CPUtime=1122.29 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 668850 0 0 0 111990 239 0 0 25 0 1 0 515258472 120676352 25107 33554432000 4194304 4587354 140736096288528 18446744073709551615 4241080 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 29462 25107 337 96 0 26024 0
[pid=27454] ppid=27452 vsize=420216 CPUtime=1122.31 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 102327 0 0 0 112203 28 0 0 25 0 1 0 515258472 430301184 100894 33554432000 4194304 4785740 140734352268960 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 105054 100894 340 145 0 102099 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=558.07 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 55807 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134530593 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=26976 CPUtime=1122.2 cores=2
/proc/27456/stat : 27456 (lingeling) R 27452 27452 26347 0 -1 4202496 8036 0 0 0 112219 1 0 0 25 0 1 0 515258472 27623424 5910 33554432000 4194304 4352236 140734989521088 18446744073709551615 4328100 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/27456/statm: 6744 5910 127 39 0 5824 0
[pid=27457] ppid=27452 vsize=80824 CPUtime=558.11 cores=6
/proc/27457/stat : 27457 (march_hi) R 27452 27452 26347 0 -1 4202496 44994 0 0 0 55805 6 0 0 25 0 1 0 515258472 82763776 17142 33554432000 4194304 4331988 140736448593520 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27457/statm: 20206 17142 173 34 0 18648 0
Current children cumulated CPU time (s) 4482.98
Current children cumulated vsize (KiB) 1128212

[startup+1182.3 s]
/proc/loadavg: 10.18 10.28 10.21 11/192 27522
/proc/meminfo: memFree=30554708/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=208504 CPUtime=1182.29 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 690584 0 0 0 117978 251 0 0 25 0 1 0 515258472 213508096 33758 33554432000 4194304 4587354 140736096288528 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 52126 33758 337 96 0 48688 0
[pid=27454] ppid=27452 vsize=420724 CPUtime=1182.31 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 102404 0 0 0 118203 28 0 0 25 0 1 0 515258472 430821376 100971 33554432000 4194304 4785740 140734352268960 18446744073709551615 4287008 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 105181 100971 340 145 0 102226 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=587.95 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 58795 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134528275 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=30048 CPUtime=1182.19 cores=2
/proc/27456/stat : 27456 (lingeling) R 27452 27452 26347 0 -1 4202496 8804 0 0 0 118218 1 0 0 25 0 1 0 515258472 30769152 6678 33554432000 4194304 4352236 140734989521088 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/27456/statm: 7512 6678 127 39 0 6592 0
[pid=27457] ppid=27452 vsize=80824 CPUtime=587.91 cores=6
/proc/27457/stat : 27457 (march_hi) R 27452 27452 26347 0 -1 4202496 45016 0 0 0 58785 6 0 0 25 0 1 0 515258472 82763776 17164 33554432000 4194304 4331988 140736448593520 18446744073709551615 4288317 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27457/statm: 20206 17164 173 34 0 18648 0
Current children cumulated CPU time (s) 4722.65
Current children cumulated vsize (KiB) 1222448



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 10.20 10.28 10.21 11/192 27523
/proc/meminfo: memFree=30584208/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=210372 CPUtime=1201.7 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 696012 0 0 0 119916 254 0 0 25 0 1 0 515258472 215420928 26097 33554432000 4194304 4587354 140736096288528 18446744073709551615 4238731 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 52593 26097 337 96 0 49155 0
[pid=27454] ppid=27452 vsize=420724 CPUtime=1201.72 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 102404 0 0 0 120144 28 0 0 25 0 1 0 515258472 430821376 100971 33554432000 4194304 4785740 140734352268960 18446744073709551615 4287096 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 105181 100971 340 145 0 102226 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=597.55 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 59755 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=30048 CPUtime=1201.6 cores=2
/proc/27456/stat : 27456 (lingeling) R 27452 27452 26347 0 -1 4202496 8804 0 0 0 120158 2 0 0 25 0 1 0 515258472 30769152 6678 33554432000 4194304 4352236 140734989521088 18446744073709551615 4329086 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/27456/statm: 7512 6678 127 39 0 6592 0
[pid=27457] ppid=27452 vsize=80824 CPUtime=597.6 cores=6
/proc/27457/stat : 27457 (march_hi) R 27452 27452 26347 0 -1 4202496 45021 0 0 0 59754 6 0 0 25 0 1 0 515258472 82763776 17169 33554432000 4194304 4331988 140736448593520 18446744073709551615 4263537 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27457/statm: 20206 17169 173 34 0 18648 0
Current children cumulated CPU time (s) 4800.17
Current children cumulated vsize (KiB) 1224316

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 10.20 10.28 10.21 11/192 27523
/proc/meminfo: memFree=30584208/32951124 swapFree=67111344/67111528
[pid=27452] ppid=27450 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/27452/stat : 27452 (ppfolio) S 27450 27452 26347 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515258472 18763776 389 33554432000 4194304 4287625 140736300842080 18446744073709551615 271550557711 0 0 4096 65536 14364327910255362047 0 0 17 4 0 0 0
/proc/27452/statm: 4581 389 341 23 0 91 0
[pid=27453] ppid=27452 vsize=210372 CPUtime=1201.7 cores=0
/proc/27453/stat : 27453 (cryptominisat) R 27452 27452 26347 0 -1 4202496 696012 0 0 0 119916 254 0 0 25 0 1 0 515258472 215420928 26097 33554432000 4194304 4587354 140736096288528 18446744073709551615 4238731 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/27453/statm: 52593 26097 337 96 0 49155 0
[pid=27454] ppid=27452 vsize=420724 CPUtime=1201.72 cores=4
/proc/27454/stat : 27454 (clasp) R 27452 27452 26347 0 -1 4202496 102404 0 0 0 120144 28 0 0 25 0 1 0 515258472 430821376 100971 33554432000 4194304 4785740 140734352268960 18446744073709551615 4287096 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/27454/statm: 105181 100971 340 145 0 102226 0
[pid=27455] ppid=27452 vsize=464024 CPUtime=597.55 cores=6
/proc/27455/stat : 27455 (TNM) R 27452 27452 26347 0 -1 4202496 5103 0 0 0 59755 0 0 0 25 0 1 0 515258472 475160576 5000 33554432000 134512640 134534564 4289625472 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27455/statm: 116006 5000 95 6 0 114672 0
[pid=27456] ppid=27452 vsize=30048 CPUtime=1201.6 cores=2
/proc/27456/stat : 27456 (lingeling) R 27452 27452 26347 0 -1 4202496 8804 0 0 0 120158 2 0 0 25 0 1 0 515258472 30769152 6678 33554432000 4194304 4352236 140734989521088 18446744073709551615 4329086 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/27456/statm: 7512 6678 127 39 0 6592 0
[pid=27457] ppid=27452 vsize=80824 CPUtime=597.6 cores=6
/proc/27457/stat : 27457 (march_hi) R 27452 27452 26347 0 -1 4202496 45021 0 0 0 59754 6 0 0 25 0 1 0 515258472 82763776 17169 33554432000 4194304 4331988 140736448593520 18446744073709551615 4263537 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/27457/statm: 20206 17169 173 34 0 18648 0
Current children cumulated CPU time (s) 4800.17
Current children cumulated vsize (KiB) 1224316

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 27452 and gives
#  childrusage.ru_utime.tv_sec=1799
#  childrusage.ru_utime.tv_usec=143488
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=36994
# CPU time returned by wait4() is 1799.18
# while last known CPU time is 4800.17
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.17
CPU user time (s): 4797.27
CPU system time (s): 2.9
CPU usage (%): 399.444
Max. virtual memory (cumulated for all children) (KiB): 1261620

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1799.14
system time used= 0.036994
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 14429
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 253
involuntary context switches= 67903

runsolver used 1.91671 second user time and 4.64729 second system time

The end

Launcher Data 

Begin job on node133 at 2011-04-29 02:23:26
IDJOB=3190814
IDBENCH=82545
IDSOLVER=1612
FILE ID=node133/3190814-1304036606
RUNJOBID= node133-1304032500-26787
PBS_JOBID= 13170319
Free space on /tmp= 71884 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v30000-c126000-S629207485-001-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3190814-1304036606/watcher-3190814-1304036606 -o /tmp/evaluation-result-3190814-1304036606/solver-3190814-1304036606 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3190814-1304036606.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 33c09da85657b6bbed23989098d6baf7
RANDOM SEED=682915647

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31025752 kB
Buffers:        666552 kB
Cached:         569580 kB
SwapCached:        124 kB
Active:         816840 kB
Inactive:       763632 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31025752 kB
SwapTotal:    67111528 kB
SwapFree:     67111344 kB
Dirty:           12488 kB
Writeback:           0 kB
AnonPages:      343904 kB
Mapped:          16724 kB
Slab:           280096 kB
PageTables:       5444 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  2967768 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71872 MiB
End job on node133 at 2011-04-29 02:43:30