Trace number 3190825

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.32 1201.71

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v30000-c126000-S1196935200-064-UNKNOWN.cnf
MD5SUMa5ac359cd8af4f1e83d886effb49a2a9
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark204.405
Satisfiable
(Un)Satisfiability was proved
Number of variables30000
Number of clauses126000
Sum of the clauses size378000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3126000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.02	This is pico-portfolio, a naive parallel portfolio
0.00/0.02	Copyright (C) 2011 Olivier Roussel
0.00/0.02	Using 4 processing units
0.00/0.02	This portfolio uses the following solvers:
0.00/0.02	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.02	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.02	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.02	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.02	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.02	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.02	Starting clasp 1.3.6 on core(s) 5
0.00/0.02	Starting TNM 2009 on core(s) 7
0.00/0.02	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.02	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3190825-1304037336/watcher-3190825-1304037336 -o /tmp/evaluation-result-3190825-1304037336/solver-3190825-1304037336 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3190825-1304037336.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.32 9.89 9.87 6/186 3382
/proc/meminfo: memFree=24934096/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) R 3380 3382 2100 0 -1 4202496 517 0 0 0 0 0 0 0 25 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 0 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0

[startup+0.22143 s]
/proc/loadavg: 10.32 9.89 9.87 6/186 3382
/proc/meminfo: memFree=24934096/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) R 3380 3382 2100 0 -1 4202496 517 0 0 0 0 0 0 0 25 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 0 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300417 s]
/proc/loadavg: 10.32 9.89 9.87 6/186 3382
/proc/meminfo: memFree=24934096/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) R 3380 3382 2100 0 -1 4202496 517 0 0 0 0 0 0 0 25 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 0 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.400397 s]
/proc/loadavg: 10.32 9.89 9.87 6/186 3382
/proc/meminfo: memFree=24934096/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) R 3380 3382 2100 0 -1 4202496 517 0 0 0 0 0 0 0 25 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 0 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700342 s]
/proc/loadavg: 10.32 9.89 9.87 6/186 3382
/proc/meminfo: memFree=24934096/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) R 3380 3382 2100 0 -1 4202496 517 0 0 0 0 0 0 0 25 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 0 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.5012 s]
/proc/loadavg: 10.32 9.89 9.87 11/192 3388
/proc/meminfo: memFree=24832988/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 23 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=51192 CPUtime=1.49 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 10873 0 0 0 148 1 0 0 20 0 1 0 515331837 52420608 8465 33554432000 4194304 4587354 140735694678720 18446744073709551615 246516530955 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 12798 8465 326 96 0 9360 0
[pid=3384] ppid=3382 vsize=28524 CPUtime=1.49 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 5886 0 0 0 148 1 0 0 25 0 1 0 515331837 29208576 4453 33554432000 4194304 4785740 140736666375504 18446744073709551615 4553068 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 7131 4453 340 145 0 4176 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=0.77 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 77 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=16176 CPUtime=1.49 cores=3
/proc/3386/stat : 3386 (lingeling) R 3382 3382 2100 0 -1 4202496 5273 0 0 0 148 1 0 0 25 0 1 0 515331837 16564224 3147 33554432000 4194304 4352236 140735559798560 18446744073709551615 4328009 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/3386/statm: 4044 3147 127 39 0 3124 0
[pid=3387] ppid=3382 vsize=24268 CPUtime=0.69 cores=7
/proc/3387/stat : 3387 (march_hi) R 3382 3382 2100 0 -1 4202496 18199 0 0 0 67 2 0 0 25 0 1 0 515331837 24850432 3412 33554432000 4194304 4331988 140734125701888 18446744073709551615 4256793 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3387/statm: 6067 3412 143 34 0 4509 0
Current children cumulated CPU time (s) 5.93
Current children cumulated vsize (KiB) 602508

[startup+3.10091 s]
/proc/loadavg: 10.32 9.89 9.87 11/192 3388
/proc/meminfo: memFree=24825052/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=88696 CPUtime=3.09 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 11885 0 0 0 307 2 0 0 25 0 1 0 515331837 90824704 9471 33554432000 4194304 4587354 140735694678720 18446744073709551615 4239479 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 22174 9471 326 96 0 18736 0
[pid=3384] ppid=3382 vsize=33888 CPUtime=3.09 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 7219 0 0 0 308 1 0 0 25 0 1 0 515331837 34701312 5786 33554432000 4194304 4785740 140736666375504 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 8472 5786 340 145 0 5517 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=1.56 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 156 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134530701 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=16176 CPUtime=3.09 cores=3
/proc/3386/stat : 3386 (lingeling) R 3382 3382 2100 0 -1 4202496 5273 0 0 0 308 1 0 0 25 0 1 0 515331837 16564224 3147 33554432000 4194304 4352236 140735559798560 18446744073709551615 4207874 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/3386/statm: 4044 3147 127 39 0 3124 0
[pid=3387] ppid=3382 vsize=66360 CPUtime=1.5 cores=7
/proc/3387/stat : 3387 (march_hi) R 3382 3382 2100 0 -1 4202496 33730 0 0 0 145 5 0 0 25 0 1 0 515331837 67952640 9195 33554432000 4194304 4331988 140734125701888 18446744073709551615 4229294 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3387/statm: 16590 9195 173 34 0 15032 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 687468

[startup+6.30032 s]
/proc/loadavg: 10.29 9.89 9.87 11/192 3389
/proc/meminfo: memFree=24785404/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=88696 CPUtime=6.29 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 13682 0 0 0 626 3 0 0 25 0 1 0 515331837 90824704 11262 33554432000 4194304 4587354 140735694678720 18446744073709551615 4238568 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 22174 11262 326 96 0 18736 0
[pid=3384] ppid=3382 vsize=47228 CPUtime=6.29 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 10544 0 0 0 627 2 0 0 25 0 1 0 515331837 48361472 9111 33554432000 4194304 4785740 140736666375504 18446744073709551615 4287042 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 11807 9111 340 145 0 8852 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=3.15 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 315 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134528913 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=16176 CPUtime=6.29 cores=3
/proc/3386/stat : 3386 (lingeling) R 3382 3382 2100 0 -1 4202496 5273 0 0 0 628 1 0 0 25 0 1 0 515331837 16564224 3147 33554432000 4194304 4352236 140735559798560 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/3386/statm: 4044 3147 127 39 0 3124 0
[pid=3387] ppid=3382 vsize=67332 CPUtime=3.09 cores=7
/proc/3387/stat : 3387 (march_hi) R 3382 3382 2100 0 -1 4202496 38392 0 0 0 304 5 0 0 25 0 1 0 515331837 68947968 13857 33554432000 4194304 4331988 140734125701888 18446744073709551615 4228998 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3387/statm: 16833 13857 173 34 0 15275 0
Current children cumulated CPU time (s) 25.11
Current children cumulated vsize (KiB) 701780

[startup+12.7011 s]
/proc/loadavg: 10.35 9.91 9.87 12/192 3389
/proc/meminfo: memFree=24738244/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=473296 CPUtime=12.69 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 18995 0 0 0 1264 5 0 0 25 0 1 0 515331837 484655104 10604 33554432000 4194304 4587354 140735694678720 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 118324 10604 331 96 0 114886 0
[pid=3384] ppid=3382 vsize=72132 CPUtime=12.69 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 16737 0 0 0 1265 4 0 0 25 0 1 0 515331837 73863168 15304 33554432000 4194304 4785740 140736666375504 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 18033 15309 340 145 0 15078 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=6.31 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 631 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=21708 CPUtime=12.68 cores=3
/proc/3386/stat : 3386 (lingeling) R 3382 3382 2100 0 -1 4202496 6720 0 0 0 1267 1 0 0 25 0 1 0 515331837 22228992 4594 33554432000 4194304 4352236 140735559798560 18446744073709551615 4328100 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/3386/statm: 5427 4594 127 39 0 4507 0
[pid=3387] ppid=3382 vsize=71720 CPUtime=6.29 cores=7
/proc/3387/stat : 3387 (march_hi) R 3382 3382 2100 0 -1 4202496 39407 0 0 0 624 5 0 0 25 0 1 0 515331837 73441280 14872 33554432000 4194304 4331988 140734125701888 18446744073709551615 4211493 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3387/statm: 17930 14872 173 34 0 16372 0
Current children cumulated CPU time (s) 50.66
Current children cumulated vsize (KiB) 1121204

[startup+25.5068 s]
/proc/loadavg: 10.27 9.91 9.87 11/192 3389
/proc/meminfo: memFree=24657532/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=105244 CPUtime=25.49 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 56340 0 0 0 2538 11 0 0 25 0 1 0 515331837 107769856 21319 33554432000 4194304 4587354 140735694678720 18446744073709551615 4388190 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 26311 21319 337 96 0 22873 0
[pid=3384] ppid=3382 vsize=101268 CPUtime=25.49 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 23953 0 0 0 2543 6 0 0 25 0 1 0 515331837 103698432 22520 33554432000 4194304 4785740 140736666375504 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 25317 22520 340 145 0 22362 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=12.7 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 1270 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134528275 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=23868 CPUtime=25.49 cores=3
/proc/3386/stat : 3386 (lingeling) R 3382 3382 2100 0 -1 4202496 7260 0 0 0 2548 1 0 0 25 0 1 0 515331837 24440832 5134 33554432000 4194304 4352236 140735559798560 18446744073709551615 4207791 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/3386/statm: 5967 5134 127 39 0 5047 0
[pid=3387] ppid=3382 vsize=74888 CPUtime=12.64 cores=7
/proc/3387/stat : 3387 (march_hi) R 3382 3382 2100 0 -1 4202496 40180 0 0 0 1259 5 0 0 25 0 1 0 515331837 76685312 15645 33554432000 4194304 4331988 140734125701888 18446744073709551615 4288532 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3387/statm: 18722 15645 173 34 0 17164 0
Current children cumulated CPU time (s) 101.81
Current children cumulated vsize (KiB) 787616

[startup+51.1011 s]
/proc/loadavg: 10.18 9.92 9.88 11/192 3390
/proc/meminfo: memFree=24662636/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=479332 CPUtime=51.09 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 86165 0 0 0 5088 21 0 0 25 0 1 0 515331837 490835968 11661 33554432000 4194304 4587354 140735694678720 18446744073709551615 246516530955 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 119833 11661 337 96 0 116395 0
[pid=3384] ppid=3382 vsize=129300 CPUtime=51.09 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 30940 0 0 0 5100 9 0 0 25 0 1 0 515331837 132403200 29507 33554432000 4194304 4785740 140736666375504 18446744073709551615 4287030 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 32325 29507 340 145 0 29370 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=25.41 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 2541 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134528913 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=26828 CPUtime=51.08 cores=3

################
# More data... #
################

/proc/3387/statm: 20108 17055 173 34 0 18550 0
Current children cumulated CPU time (s) 3764.11
Current children cumulated vsize (KiB) 1484872

[startup+1002.31 s]
/proc/loadavg: 10.22 10.24 10.11 11/192 3450
/proc/meminfo: memFree=24029856/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=509748 CPUtime=1002.32 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 626366 0 0 0 99992 240 0 0 25 0 1 0 515331837 521981952 21340 33554432000 4194304 4587354 140735694678720 18446744073709551615 4238294 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 127437 21340 337 96 0 123999 0
[pid=3384] ppid=3382 vsize=369156 CPUtime=1002.33 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 89586 0 0 0 100205 28 0 0 25 0 1 0 515331837 378015744 88153 33554432000 4194304 4785740 140736666375504 18446744073709551615 4287244 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 92289 88153 340 145 0 89334 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=498.42 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 49842 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134528661 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=31836 CPUtime=1002.31 cores=3
/proc/3386/stat : 3386 (lingeling) R 3382 3382 2100 0 -1 4202496 9252 0 0 0 100230 1 0 0 25 0 1 0 515331837 32600064 7126 33554432000 4194304 4352236 140735559798560 18446744073709551615 4328105 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/3386/statm: 7959 7126 127 39 0 7039 0
[pid=3387] ppid=3382 vsize=80564 CPUtime=498.42 cores=7
/proc/3387/stat : 3387 (march_hi) R 3382 3382 2100 0 -1 4202496 41623 0 0 0 49835 7 0 0 25 0 1 0 515331837 82497536 17088 33554432000 4194304 4331988 140734125701888 18446744073709551615 4226434 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3387/statm: 20141 17088 173 34 0 18583 0
Current children cumulated CPU time (s) 4003.8
Current children cumulated vsize (KiB) 1473652

[startup+1062.3 s]
/proc/loadavg: 10.37 10.29 10.14 11/192 3452
/proc/meminfo: memFree=23979684/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=134220 CPUtime=1062.32 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 650568 0 0 0 105979 253 0 0 25 0 1 0 515331837 137441280 20371 33554432000 4194304 4587354 140735694678720 18446744073709551615 4238317 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 33555 20371 337 96 0 30117 0
[pid=3384] ppid=3382 vsize=377584 CPUtime=1062.32 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 91616 0 0 0 106203 29 0 0 25 0 1 0 515331837 386646016 90183 33554432000 4194304 4785740 140736666375504 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 94396 90183 340 145 0 91441 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=528.3 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 52830 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134530701 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=31836 CPUtime=1062.3 cores=3
/proc/3386/stat : 3386 (lingeling) R 3382 3382 2100 0 -1 4202496 9252 0 0 0 106229 1 0 0 25 0 1 0 515331837 32600064 7126 33554432000 4194304 4352236 140735559798560 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/3386/statm: 7959 7126 127 39 0 7039 0
[pid=3387] ppid=3382 vsize=80696 CPUtime=528.21 cores=7
/proc/3387/stat : 3387 (march_hi) R 3382 3382 2100 0 -1 4202496 41656 0 0 0 52814 7 0 0 25 0 1 0 515331837 82632704 17121 33554432000 4194304 4331988 140734125701888 18446744073709551615 4211493 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3387/statm: 20174 17121 173 34 0 18616 0
Current children cumulated CPU time (s) 4243.45
Current children cumulated vsize (KiB) 1106684

[startup+1122.3 s]
/proc/loadavg: 10.24 10.27 10.14 11/192 3453
/proc/meminfo: memFree=23975896/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=519156 CPUtime=1122.32 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 691623 0 0 0 111964 268 0 0 25 0 1 0 515331837 531615744 16147 33554432000 4194304 4587354 140735694678720 18446744073709551615 4238597 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 129789 16147 337 96 0 126351 0
[pid=3384] ppid=3382 vsize=399152 CPUtime=1122.32 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 96920 0 0 0 112201 31 0 0 25 0 1 0 515331837 408731648 95487 33554432000 4194304 4785740 140736666375504 18446744073709551615 4291806 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 99788 95487 340 145 0 96833 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=558.11 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 55811 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134528960 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=31836 CPUtime=1122.31 cores=3
/proc/3386/stat : 3386 (lingeling) R 3382 3382 2100 0 -1 4202496 9252 0 0 0 112230 1 0 0 25 0 1 0 515331837 32600064 7126 33554432000 4194304 4352236 140735559798560 18446744073709551615 4328475 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/3386/statm: 7959 7126 127 39 0 7039 0
[pid=3387] ppid=3382 vsize=80828 CPUtime=558.07 cores=7
/proc/3387/stat : 3387 (march_hi) R 3382 3382 2100 0 -1 4202496 41678 0 0 0 55800 7 0 0 25 0 1 0 515331837 82767872 17143 33554432000 4194304 4331988 140734125701888 18446744073709551615 4211525 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3387/statm: 20207 17143 173 34 0 18649 0
Current children cumulated CPU time (s) 4483.13
Current children cumulated vsize (KiB) 1513320

[startup+1182.3 s]
/proc/loadavg: 10.35 10.29 10.15 11/192 3455
/proc/meminfo: memFree=23888404/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=512756 CPUtime=1182.32 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 716792 0 0 0 117949 283 0 0 25 0 1 0 515331837 525062144 28166 33554432000 4194304 4587354 140735694678720 18446744073709551615 246516530955 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 128189 28166 337 96 0 124751 0
[pid=3384] ppid=3382 vsize=406196 CPUtime=1182.32 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 98691 0 0 0 118201 31 0 0 25 0 1 0 515331837 415944704 97258 33554432000 4194304 4785740 140736666375504 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 101549 97258 340 145 0 98594 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=587.96 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 58796 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134530484 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=31836 CPUtime=1182.31 cores=3
/proc/3386/stat : 3386 (lingeling) R 3382 3382 2100 0 -1 4202496 9252 0 0 0 118230 1 0 0 25 0 1 0 515331837 32600064 7126 33554432000 4194304 4352236 140735559798560 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/3386/statm: 7959 7126 127 39 0 7039 0
[pid=3387] ppid=3382 vsize=80828 CPUtime=587.9 cores=7
/proc/3387/stat : 3387 (march_hi) R 3382 3382 2100 0 -1 4202496 41700 0 0 0 58783 7 0 0 25 0 1 0 515331837 82767872 17165 33554432000 4194304 4331988 140734125701888 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3387/statm: 20207 17165 173 34 0 18649 0
Current children cumulated CPU time (s) 4722.81
Current children cumulated vsize (KiB) 1513964



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 10.44 10.32 10.16 11/192 3455
/proc/meminfo: memFree=23941460/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=546444 CPUtime=1201.73 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 722405 0 0 0 119886 287 0 0 25 0 1 0 515331837 559558656 16147 33554432000 4194304 4587354 140735694678720 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 136611 16147 337 96 0 133173 0
[pid=3384] ppid=3382 vsize=406196 CPUtime=1201.72 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 98694 0 0 0 120141 31 0 0 25 0 1 0 515331837 415944704 97261 33554432000 4194304 4785740 140736666375504 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 101549 97261 340 145 0 98594 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=597.61 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 59761 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134525904 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=31836 CPUtime=1201.71 cores=3
/proc/3386/stat : 3386 (lingeling) R 3382 3382 2100 0 -1 4202496 9252 0 0 0 120170 1 0 0 25 0 1 0 515331837 32600064 7126 33554432000 4194304 4352236 140735559798560 18446744073709551615 4328400 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/3386/statm: 7959 7126 127 39 0 7039 0
[pid=3387] ppid=3382 vsize=80960 CPUtime=597.55 cores=7
/proc/3387/stat : 3387 (march_hi) R 3382 3382 2100 0 -1 4202496 41707 0 0 0 59748 7 0 0 25 0 1 0 515331837 82903040 17172 33554432000 4194304 4331988 140734125701888 18446744073709551615 4237180 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3387/statm: 20240 17172 173 34 0 18682 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 1547784

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 10.44 10.32 10.16 11/192 3455
/proc/meminfo: memFree=23941460/32951124 swapFree=67111352/67111528
[pid=3382] ppid=3380 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/3382/stat : 3382 (ppfolio) S 3380 3382 2100 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 515331836 18763776 389 33554432000 4194304 4287625 140735778418080 18446744073709551615 246516854287 0 0 4096 65536 15268706735548792831 0 0 17 1 0 0 0
/proc/3382/statm: 4581 389 341 23 0 91 0
[pid=3383] ppid=3382 vsize=546444 CPUtime=1201.73 cores=1
/proc/3383/stat : 3383 (cryptominisat) R 3382 3382 2100 0 -1 4202496 722405 0 0 0 119886 287 0 0 25 0 1 0 515331837 559558656 16147 33554432000 4194304 4587354 140735694678720 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/3383/statm: 136611 16147 337 96 0 133173 0
[pid=3384] ppid=3382 vsize=406196 CPUtime=1201.72 cores=5
/proc/3384/stat : 3384 (clasp) R 3382 3382 2100 0 -1 4202496 98694 0 0 0 120141 31 0 0 25 0 1 0 515331837 415944704 97261 33554432000 4194304 4785740 140736666375504 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/3384/statm: 101549 97261 340 145 0 98594 0
[pid=3385] ppid=3382 vsize=464024 CPUtime=597.61 cores=7
/proc/3385/stat : 3385 (TNM) R 3382 3382 2100 0 -1 4202496 5104 0 0 0 59761 0 0 0 25 0 1 0 515331837 475160576 5001 33554432000 134512640 134534564 4286656256 18446744073709551615 134525904 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3385/statm: 116006 5001 95 6 0 114672 0
[pid=3386] ppid=3382 vsize=31836 CPUtime=1201.71 cores=3
/proc/3386/stat : 3386 (lingeling) R 3382 3382 2100 0 -1 4202496 9252 0 0 0 120170 1 0 0 25 0 1 0 515331837 32600064 7126 33554432000 4194304 4352236 140735559798560 18446744073709551615 4328400 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/3386/statm: 7959 7126 127 39 0 7039 0
[pid=3387] ppid=3382 vsize=80960 CPUtime=597.55 cores=7
/proc/3387/stat : 3387 (march_hi) R 3382 3382 2100 0 -1 4202496 41707 0 0 0 59748 7 0 0 25 0 1 0 515331837 82903040 17172 33554432000 4194304 4331988 140734125701888 18446744073709551615 4237180 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/3387/statm: 20240 17172 173 34 0 18682 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 1547784

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 3382 and gives
#  childrusage.ru_utime.tv_sec=2998
#  childrusage.ru_utime.tv_usec=62225
#  childrusage.ru_stime.tv_sec=2
#  childrusage.ru_stime.tv_usec=980546
# CPU time returned by wait4() is 3001.04
# while last known CPU time is 4800.32
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.32
CPU user time (s): 4797.06
CPU system time (s): 3.26
CPU usage (%): 399.456
Max. virtual memory (cumulated for all children) (KiB): 1651548

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2998.06
system time used= 2.98055
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 773885
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 255
involuntary context switches= 48810

runsolver used 1.78573 second user time and 4.77827 second system time

The end

Launcher Data 

Begin job on node134 at 2011-04-29 02:35:36
IDJOB=3190825
IDBENCH=82556
IDSOLVER=1612
FILE ID=node134/3190825-1304037336
RUNJOBID= node134-1304036955-3307
PBS_JOBID= 13170318
Free space on /tmp= 71572 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v30000-c126000-S1196935200-064-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3190825-1304037336/watcher-3190825-1304037336 -o /tmp/evaluation-result-3190825-1304037336/solver-3190825-1304037336 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3190825-1304037336.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= a5ac359cd8af4f1e83d886effb49a2a9
RANDOM SEED=1068574923

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24934376 kB
Buffers:       1470128 kB
Cached:        5783136 kB
SwapCached:         64 kB
Active:        1575976 kB
Inactive:      5919064 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24934376 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:           12052 kB
Writeback:           0 kB
AnonPages:      241332 kB
Mapped:          17080 kB
Slab:           458292 kB
PageTables:       5752 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  1973372 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71560 MiB
End job on node134 at 2011-04-29 02:55:40