Trace number 3190925

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.32 1201.71

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v25000-c105000-S269939581-066-UNKNOWN.cnf
MD5SUM3d91d29f74f04e1ffeedae8425f55746
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark25.5211
Satisfiable
(Un)Satisfiability was proved
Number of variables25000
Number of clauses105000
Sum of the clauses size315000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3105000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.80	This is pico-portfolio, a naive parallel portfolio
0.00/0.80	Copyright (C) 2011 Olivier Roussel
0.00/0.80	Using 4 processing units
0.00/0.80	This portfolio uses the following solvers:
0.00/0.80	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.80	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.80	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.80	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.80	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.80	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.80	Starting clasp 1.3.6 on core(s) 4
0.00/0.80	Starting TNM 2009 on core(s) 6
0.00/0.80	Starting lingeling 276-6264d55-100731 on core(s) 2
0.00/0.80	Starting march_hi 2009 on core(s) 6

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3190925-1304045240/watcher-3190925-1304045240 -o /tmp/evaluation-result-3190925-1304045240/solver-3190925-1304045240 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3190925-1304045240.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.17 10.20 10.18 6/182 28607
/proc/meminfo: memFree=26505904/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) R 28605 28607 25547 0 -1 4202496 520 0 0 0 0 0 0 0 25 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 0 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0

[startup+0.431272 s]
/proc/loadavg: 10.17 10.20 10.18 6/182 28607
/proc/meminfo: memFree=26505904/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 520 0 0 0 0 0 0 0 23 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.500261 s]
/proc/loadavg: 10.17 10.20 10.18 6/182 28607
/proc/meminfo: memFree=26505904/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 520 0 0 0 0 0 0 0 23 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.600242 s]
/proc/loadavg: 10.17 10.20 10.18 6/182 28607
/proc/meminfo: memFree=26505904/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 520 0 0 0 0 0 0 0 23 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700223 s]
/proc/loadavg: 10.17 10.20 10.18 6/182 28607
/proc/meminfo: memFree=26505904/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 520 0 0 0 0 0 0 0 23 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50707 s]
/proc/loadavg: 10.23 10.22 10.18 11/188 28613
/proc/meminfo: memFree=26428480/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 520 0 0 0 0 0 0 0 23 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=47168 CPUtime=1.49 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 9422 0 0 0 148 1 0 0 20 0 1 0 496939448 48300032 7396 33554432000 4194304 4587354 140735996367344 18446744073709551615 4238282 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 11792 7396 326 96 0 8354 0
[pid=28609] ppid=28607 vsize=26820 CPUtime=1.49 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 5369 0 0 0 149 0 0 0 20 0 1 0 496939448 27463680 4023 33554432000 4194304 4785740 140735862560064 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 6705 4023 340 145 0 3750 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=0.69 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 69 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=11500 CPUtime=1.49 cores=2
/proc/28611/stat : 28611 (lingeling) R 28607 28607 25547 0 -1 4202496 3770 0 0 0 149 0 0 0 25 0 1 0 496939448 11776000 1909 33554432000 4194304 4352236 140734794909776 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28611/statm: 2875 1909 127 39 0 1955 0
[pid=28612] ppid=28607 vsize=25292 CPUtime=0.78 cores=6
/proc/28612/stat : 28612 (march_hi) R 28607 28607 25547 0 -1 4202496 21681 0 0 0 76 2 0 0 25 0 1 0 496939448 25899008 3840 33554432000 4194304 4331988 140737321089920 18446744073709551615 4253900 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28612/statm: 6323 3840 143 34 0 4765 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 591000

[startup+3.10078 s]
/proc/loadavg: 10.23 10.22 10.18 11/188 28613
/proc/meminfo: memFree=26399092/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=84672 CPUtime=3.09 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 10513 0 0 0 308 1 0 0 25 0 1 0 496939448 86704128 8483 33554432000 4194304 4587354 140735996367344 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 21168 8483 326 96 0 17730 0
[pid=28609] ppid=28607 vsize=34088 CPUtime=3.08 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 7189 0 0 0 308 0 0 0 25 0 1 0 496939448 34906112 5843 33554432000 4194304 4785740 140735862560064 18446744073709551615 228095118240 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 8522 5843 340 145 0 5567 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=1.49 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 149 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134528661 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=11500 CPUtime=3.09 cores=2
/proc/28611/stat : 28611 (lingeling) R 28607 28607 25547 0 -1 4202496 3770 0 0 0 309 0 0 0 25 0 1 0 496939448 11776000 1909 33554432000 4194304 4352236 140734794909776 18446744073709551615 4329328 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28611/statm: 2875 1909 127 39 0 1955 0
[pid=28612] ppid=28607 vsize=56516 CPUtime=1.57 cores=6
/proc/28612/stat : 28612 (march_hi) R 28607 28607 25547 0 -1 4202496 34562 0 0 0 153 4 0 0 25 0 1 0 496939448 57872384 11408 33554432000 4194304 4331988 140737321089920 18446744073709551615 4229042 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28612/statm: 14129 11408 173 34 0 12571 0
Current children cumulated CPU time (s) 12.32
Current children cumulated vsize (KiB) 666996

[startup+6.30119 s]
/proc/loadavg: 10.21 10.21 10.18 11/188 28613
/proc/meminfo: memFree=26383604/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=84672 CPUtime=6.29 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 12401 0 0 0 627 2 0 0 25 0 1 0 496939448 86704128 10362 33554432000 4194304 4587354 140735996367344 18446744073709551615 228095147787 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 21168 10362 326 96 0 17730 0
[pid=28609] ppid=28607 vsize=48640 CPUtime=6.29 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 10788 0 0 0 628 1 0 0 25 0 1 0 496939448 49807360 9442 33554432000 4194304 4785740 140735862560064 18446744073709551615 4553106 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 12160 9442 340 145 0 9205 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=3.09 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 309 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134528824 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=11500 CPUtime=6.29 cores=2
/proc/28611/stat : 28611 (lingeling) R 28607 28607 25547 0 -1 4202496 3770 0 0 0 629 0 0 0 25 0 1 0 496939448 11776000 1909 33554432000 4194304 4352236 140734794909776 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28611/statm: 2875 1909 127 39 0 1955 0
[pid=28612] ppid=28607 vsize=58016 CPUtime=3.15 cores=6
/proc/28612/stat : 28612 (march_hi) R 28607 28607 25547 0 -1 4202496 34901 0 0 0 311 4 0 0 25 0 1 0 496939448 59408384 11747 33554432000 4194304 4331988 140737321089920 18446744073709551615 4299872 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28612/statm: 14504 11747 173 34 0 12946 0
Current children cumulated CPU time (s) 25.11
Current children cumulated vsize (KiB) 683048

[startup+12.701 s]
/proc/loadavg: 10.20 10.21 10.18 11/188 28614
/proc/meminfo: memFree=26330636/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=192560 CPUtime=12.69 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 36267 0 0 0 1263 6 0 0 25 0 1 0 496939448 197181440 28751 33554432000 4194304 4587354 140735996367344 18446744073709551615 4388336 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 48140 28751 337 96 0 44702 0
[pid=28609] ppid=28607 vsize=73116 CPUtime=12.69 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 16914 0 0 0 1267 2 0 0 25 0 1 0 496939448 74870784 15568 33554432000 4194304 4785740 140735862560064 18446744073709551615 4279309 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 18279 15568 340 145 0 15324 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=6.29 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 629 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=16344 CPUtime=12.68 cores=2
/proc/28611/stat : 28611 (lingeling) R 28607 28607 25547 0 -1 4202496 5113 0 0 0 1268 0 0 0 25 0 1 0 496939448 16736256 3252 33554432000 4194304 4352236 140734794909776 18446744073709551615 4207874 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28611/statm: 4086 3252 127 39 0 3166 0
[pid=28612] ppid=28607 vsize=63160 CPUtime=6.32 cores=6
/proc/28612/stat : 28612 (march_hi) R 28607 28607 25547 0 -1 4202496 36081 0 0 0 627 5 0 0 25 0 1 0 496939448 64675840 12927 33554432000 4194304 4331988 140737321089920 18446744073709551615 4297607 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28612/statm: 15790 12927 173 34 0 14232 0
Current children cumulated CPU time (s) 50.67
Current children cumulated vsize (KiB) 825400

[startup+25.5006 s]
/proc/loadavg: 10.16 10.20 10.18 11/188 28614
/proc/meminfo: memFree=26311276/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=123436 CPUtime=25.49 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 55125 0 0 0 2539 10 0 0 25 0 1 0 496939448 126398464 10732 33554432000 4194304 4587354 140735996367344 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 30859 10732 337 96 0 27421 0
[pid=28609] ppid=28607 vsize=84272 CPUtime=25.49 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 19684 0 0 0 2546 3 0 0 25 0 1 0 496939448 86294528 18338 33554432000 4194304 4785740 140735862560064 18446744073709551615 4553148 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 21068 18338 340 145 0 18113 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=12.63 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 1263 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134528275 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=17756 CPUtime=25.48 cores=2
/proc/28611/stat : 28611 (lingeling) R 28607 28607 25547 0 -1 4202496 5466 0 0 0 2548 0 0 0 25 0 1 0 496939448 18182144 3605 33554432000 4194304 4352236 140734794909776 18446744073709551615 4328116 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28611/statm: 4439 3605 127 39 0 3519 0
[pid=28612] ppid=28607 vsize=64348 CPUtime=12.72 cores=6
/proc/28612/stat : 28612 (march_hi) R 28607 28607 25547 0 -1 4202496 36378 0 0 0 1267 5 0 0 25 0 1 0 496939448 65892352 13224 33554432000 4194304 4331988 140737321089920 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28612/statm: 16087 13224 173 34 0 14529 0
Current children cumulated CPU time (s) 101.81
Current children cumulated vsize (KiB) 770032

[startup+51.1009 s]
/proc/loadavg: 10.15 10.20 10.18 11/188 28615
/proc/meminfo: memFree=26149124/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=1333344 CPUtime=51.09 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 113032 0 0 0 5086 23 0 0 25 0 1 0 496939448 1365344256 41446 33554432000 4194304 4587354 140735996367344 18446744073709551615 4388350 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 333336 41446 337 96 0 329898 0
[pid=28609] ppid=28607 vsize=112540 CPUtime=51.09 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 26588 0 0 0 5104 5 0 0 25 0 1 0 496939448 115240960 25242 33554432000 4194304 4785740 140735862560064 18446744073709551615 4304236 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 28135 25242 340 145 0 25180 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=25.38 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 2538 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134525904 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=24660 CPUtime=51.08 cores=2

################
# More data... #
################

/proc/28612/statm: 17275 14432 173 34 0 15717 0
Current children cumulated CPU time (s) 3764.08
Current children cumulated vsize (KiB) 1038548

[startup+1002.3 s]
/proc/loadavg: 5.05 5.68 7.62 6/177 28698
/proc/meminfo: memFree=26118448/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=1239876 CPUtime=1002.32 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 614658 0 0 0 100002 230 0 0 25 0 1 0 496939448 1269633024 20936 33554432000 4194304 4587354 140735996367344 18446744073709551615 4237695 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 309969 20936 337 96 0 306531 0
[pid=28609] ppid=28607 vsize=315812 CPUtime=1002.28 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 76307 0 0 0 100206 22 0 0 25 0 1 0 496939448 323391488 74961 33554432000 4194304 4785740 140735862560064 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 78953 74961 340 145 0 75998 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=498.41 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 49841 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134529675 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=31568 CPUtime=1002.29 cores=2
/proc/28611/stat : 28611 (lingeling) R 28607 28607 25547 0 -1 4202496 10967 0 0 0 100227 2 0 0 25 0 1 0 496939448 32325632 7058 33554432000 4194304 4352236 140734794909776 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28611/statm: 7892 7058 127 39 0 6972 0
[pid=28612] ppid=28607 vsize=69232 CPUtime=498.45 cores=6
/proc/28612/stat : 28612 (march_hi) R 28607 28607 25547 0 -1 4202496 37604 0 0 0 49840 5 0 0 25 0 1 0 496939448 70893568 14450 33554432000 4194304 4331988 140737321089920 18446744073709551615 4226530 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28612/statm: 17308 14450 173 34 0 15750 0
Current children cumulated CPU time (s) 4003.75
Current children cumulated vsize (KiB) 2136708

[startup+1062.3 s]
/proc/loadavg: 5.06 5.57 7.46 6/177 28700
/proc/meminfo: memFree=26126060/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=1231428 CPUtime=1062.32 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 641629 0 0 0 105986 246 0 0 25 0 1 0 496939448 1260982272 15875 33554432000 4194304 4587354 140735996367344 18446744073709551615 4238719 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 307857 15875 337 96 0 304419 0
[pid=28609] ppid=28607 vsize=329660 CPUtime=1062.28 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 79680 0 0 0 106205 23 0 0 25 0 1 0 496939448 337571840 78334 33554432000 4194304 4785740 140735862560064 18446744073709551615 4553423 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 82415 78334 340 145 0 79460 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=528.21 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 52821 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134530456 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=29520 CPUtime=1062.29 cores=2
/proc/28611/stat : 28611 (lingeling) R 28607 28607 25547 0 -1 4202496 10967 0 0 0 106226 3 0 0 25 0 1 0 496939448 30228480 6546 33554432000 4194304 4352236 140734794909776 18446744073709551615 4328090 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28611/statm: 7380 6546 127 39 0 6460 0
[pid=28612] ppid=28607 vsize=69364 CPUtime=528.33 cores=6
/proc/28612/stat : 28612 (march_hi) R 28607 28607 25547 0 -1 4202496 37623 0 0 0 52828 5 0 0 25 0 1 0 496939448 71028736 14469 33554432000 4194304 4331988 140737321089920 18446744073709551615 4226434 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28612/statm: 17341 14469 173 34 0 15783 0
Current children cumulated CPU time (s) 4243.43
Current children cumulated vsize (KiB) 2140192

[startup+1122.3 s]
/proc/loadavg: 5.06 5.48 7.31 6/177 28701
/proc/meminfo: memFree=26079472/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=1238376 CPUtime=1122.33 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 666893 0 0 0 111975 258 0 0 25 0 1 0 496939448 1268097024 24798 33554432000 4194304 4587354 140735996367344 18446744073709551615 228095147787 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 309594 24798 337 96 0 306156 0
[pid=28609] ppid=28607 vsize=339600 CPUtime=1122.29 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 82164 0 0 0 112206 23 0 0 25 0 1 0 496939448 347750400 80818 33554432000 4194304 4785740 140735862560064 18446744073709551615 4287204 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 84900 80818 340 145 0 81945 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=558.1 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 55810 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134530619 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=29520 CPUtime=1122.3 cores=2
/proc/28611/stat : 28611 (lingeling) R 28607 28607 25547 0 -1 4202496 11479 0 0 0 112227 3 0 0 25 0 1 0 496939448 30228480 6546 33554432000 4194304 4352236 140734794909776 18446744073709551615 4329060 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28611/statm: 7380 6546 127 39 0 6460 0
[pid=28612] ppid=28607 vsize=69364 CPUtime=558.13 cores=6
/proc/28612/stat : 28612 (march_hi) R 28607 28607 25547 0 -1 4202496 37642 0 0 0 55808 5 0 0 25 0 1 0 496939448 71028736 14488 33554432000 4194304 4331988 140737321089920 18446744073709551615 4263152 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28612/statm: 17341 14488 173 34 0 15783 0
Current children cumulated CPU time (s) 4483.15
Current children cumulated vsize (KiB) 2157080

[startup+1182.3 s]
/proc/loadavg: 5.06 5.40 7.17 6/177 28703
/proc/meminfo: memFree=26086472/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=1239696 CPUtime=1182.33 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 693607 0 0 0 117961 272 0 0 25 0 1 0 496939448 1269448704 18091 33554432000 4194304 4587354 140735996367344 18446744073709551615 4238271 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 309924 18091 337 96 0 306486 0
[pid=28609] ppid=28607 vsize=361452 CPUtime=1182.28 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 87546 0 0 0 118203 25 0 0 25 0 1 0 496939448 370126848 86200 33554432000 4194304 4785740 140735862560064 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 90363 86200 340 145 0 87408 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=587.9 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 58790 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134528659 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=29520 CPUtime=1182.29 cores=2
/proc/28611/stat : 28611 (lingeling) R 28607 28607 25547 0 -1 4202496 11991 0 0 0 118226 3 0 0 25 0 1 0 496939448 30228480 6546 33554432000 4194304 4352236 140734794909776 18446744073709551615 4207882 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28611/statm: 7380 6546 127 39 0 6460 0
[pid=28612] ppid=28607 vsize=69496 CPUtime=588 cores=6
/proc/28612/stat : 28612 (march_hi) R 28607 28607 25547 0 -1 4202496 37660 0 0 0 58795 5 0 0 25 0 1 0 496939448 71163904 14506 33554432000 4194304 4331988 140737321089920 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28612/statm: 17374 14506 173 34 0 15816 0
Current children cumulated CPU time (s) 4722.8
Current children cumulated vsize (KiB) 2180384



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 5.11 5.39 7.13 6/177 28703
/proc/meminfo: memFree=26044028/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=1239696 CPUtime=1201.73 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 701116 0 0 0 119897 276 0 0 25 0 1 0 496939448 1269448704 25569 33554432000 4194304 4587354 140735996367344 18446744073709551615 4230266 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 309924 25569 337 96 0 306486 0
[pid=28609] ppid=28607 vsize=370424 CPUtime=1201.69 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 89791 0 0 0 120143 26 0 0 25 0 1 0 496939448 379314176 88445 33554432000 4194304 4785740 140735862560064 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 92606 88445 340 145 0 89651 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=597.58 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 59758 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134528877 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=31568 CPUtime=1201.7 cores=2
/proc/28611/stat : 28611 (lingeling) R 28607 28607 25547 0 -1 4202496 12503 0 0 0 120167 3 0 0 25 0 1 0 496939448 32325632 7058 33554432000 4194304 4352236 140734794909776 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28611/statm: 7892 7058 127 39 0 6972 0
[pid=28612] ppid=28607 vsize=69496 CPUtime=597.62 cores=6
/proc/28612/stat : 28612 (march_hi) R 28607 28607 25547 0 -1 4202496 37664 0 0 0 59757 5 0 0 25 0 1 0 496939448 71163904 14510 33554432000 4194304 4331988 140737321089920 18446744073709551615 4228841 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28612/statm: 17374 14510 173 34 0 15816 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 2191404

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 5.11 5.39 7.13 6/177 28703
/proc/meminfo: memFree=26044028/32951124 swapFree=67111524/67111528
[pid=28607] ppid=28605 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/28607/stat : 28607 (ppfolio) S 28605 28607 25547 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 496939448 18763776 390 33554432000 4194304 4287625 140734694965216 18446744073709551615 228095471119 0 0 4096 65536 4577610233232228351 0 0 17 2 0 0 0
/proc/28607/statm: 4581 390 341 23 0 91 0
[pid=28608] ppid=28607 vsize=1239696 CPUtime=1201.73 cores=0
/proc/28608/stat : 28608 (cryptominisat) R 28607 28607 25547 0 -1 4202496 701116 0 0 0 119897 276 0 0 25 0 1 0 496939448 1269448704 25569 33554432000 4194304 4587354 140735996367344 18446744073709551615 4230266 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28608/statm: 309924 25569 337 96 0 306486 0
[pid=28609] ppid=28607 vsize=370424 CPUtime=1201.69 cores=4
/proc/28609/stat : 28609 (clasp) R 28607 28607 25547 0 -1 4202496 89791 0 0 0 120143 26 0 0 25 0 1 0 496939448 379314176 88445 33554432000 4194304 4785740 140735862560064 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/28609/statm: 92606 88445 340 145 0 89651 0
[pid=28610] ppid=28607 vsize=461896 CPUtime=597.58 cores=6
/proc/28610/stat : 28610 (TNM) R 28607 28607 25547 0 -1 4202496 4293 0 0 0 59758 0 0 0 25 0 1 0 496939448 472981504 4190 33554432000 134512640 134534564 4288105504 18446744073709551615 134528877 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28610/statm: 115474 4190 95 6 0 114140 0
[pid=28611] ppid=28607 vsize=31568 CPUtime=1201.7 cores=2
/proc/28611/stat : 28611 (lingeling) R 28607 28607 25547 0 -1 4202496 12503 0 0 0 120167 3 0 0 25 0 1 0 496939448 32325632 7058 33554432000 4194304 4352236 140734794909776 18446744073709551615 4327902 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/28611/statm: 7892 7058 127 39 0 6972 0
[pid=28612] ppid=28607 vsize=69496 CPUtime=597.62 cores=6
/proc/28612/stat : 28612 (march_hi) R 28607 28607 25547 0 -1 4202496 37664 0 0 0 59757 5 0 0 25 0 1 0 496939448 71163904 14510 33554432000 4194304 4331988 140737321089920 18446744073709551615 4228841 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/28612/statm: 17374 14510 173 34 0 15816 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 2191404

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 28607 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=2999
# CPU time returned by wait4() is 0.003998
# while last known CPU time is 4800.32
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.32
CPU user time (s): 4797.22
CPU system time (s): 3.1
CPU usage (%): 399.457
Max. virtual memory (cumulated for all children) (KiB): 3351100

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.000999
system time used= 0.002999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 521
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 244
involuntary context switches= 6

runsolver used 2.18167 second user time and 4.34034 second system time

The end

Launcher Data 

Begin job on node142 at 2011-04-29 04:47:20
IDJOB=3190925
IDBENCH=82656
IDSOLVER=1612
FILE ID=node142/3190925-1304045240
RUNJOBID= node142-1304041557-25982
PBS_JOBID= 13170374
Free space on /tmp= 71304 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v25000-c105000-S269939581-066-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3190925-1304045240/watcher-3190925-1304045240 -o /tmp/evaluation-result-3190925-1304045240/solver-3190925-1304045240 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3190925-1304045240.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3d91d29f74f04e1ffeedae8425f55746
RANDOM SEED=1095726848

node142.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5131.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      26506184 kB
Buffers:       1645968 kB
Cached:        4110996 kB
SwapCached:          4 kB
Active:         770556 kB
Inactive:      5135308 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      26506184 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           12060 kB
Writeback:           0 kB
AnonPages:      147960 kB
Mapped:          16700 kB
Slab:           475632 kB
PageTables:       4928 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   746936 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71304 MiB
End job on node142 at 2011-04-29 05:07:24