Trace number 3190845

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.4 1201.71

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v25000-c105000-S109959541-056-UNKNOWN.cnf
MD5SUM5f909806405bfbe25b7db5afcabe8fc9
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark87.7936
Satisfiable
(Un)Satisfiability was proved
Number of variables24999
Number of clauses105000
Sum of the clauses size315000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3105000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.80	This is pico-portfolio, a naive parallel portfolio
0.00/0.80	Copyright (C) 2011 Olivier Roussel
0.00/0.80	Using 4 processing units
0.00/0.80	This portfolio uses the following solvers:
0.00/0.80	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.80	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.80	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.80	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.80	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.80	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.80	Starting clasp 1.3.6 on core(s) 4
0.00/0.80	Starting TNM 2009 on core(s) 6
0.00/0.80	Starting lingeling 276-6264d55-100731 on core(s) 2
0.00/0.80	Starting march_hi 2009 on core(s) 6

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3190845-1304037925/watcher-3190845-1304037925 -o /tmp/evaluation-result-3190845-1304037925/solver-3190845-1304037925 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3190845-1304037925.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.19 6.72 8.19 1/171 8442
/proc/meminfo: memFree=15607832/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) R 8440 8442 7235 0 -1 4194304 502 0 0 0 0 0 0 0 25 0 1 0 515392996 18763776 387 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446631667 0 0 4096 65536 0 0 0 17 2 0 0 0
/proc/8442/statm: 4581 387 339 23 0 91 0

[startup+0.082508 s]
/proc/loadavg: 5.19 6.72 8.19 1/171 8442
/proc/meminfo: memFree=15607832/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) R 8440 8442 7235 0 -1 4194304 502 0 0 0 0 0 0 0 25 0 1 0 515392996 18763776 387 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446631667 0 0 4096 65536 0 0 0 17 2 0 0 0
/proc/8442/statm: 4581 387 339 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100499 s]
/proc/loadavg: 5.19 6.72 8.19 1/171 8442
/proc/meminfo: memFree=15607832/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) R 8440 8442 7235 0 -1 4194304 502 0 0 0 0 0 0 0 25 0 1 0 515392996 18763776 387 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446631667 0 0 4096 65536 0 0 0 17 2 0 0 0
/proc/8442/statm: 4581 387 339 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300449 s]
/proc/loadavg: 5.19 6.72 8.19 1/171 8442
/proc/meminfo: memFree=15607832/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) R 8440 8442 7235 0 -1 4194304 502 0 0 0 0 0 0 0 25 0 1 0 515392996 18763776 387 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446631667 0 0 4096 65536 0 0 0 17 2 0 0 0
/proc/8442/statm: 4581 387 339 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700376 s]
/proc/loadavg: 5.19 6.72 8.19 1/171 8442
/proc/meminfo: memFree=15607832/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) R 8440 8442 7235 0 -1 4194304 502 0 0 0 0 0 0 0 25 0 1 0 515392996 18763776 387 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446631667 0 0 4096 65536 0 0 0 17 2 0 0 0
/proc/8442/statm: 4581 387 339 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50123 s]
/proc/loadavg: 5.18 6.69 8.17 6/177 8448
/proc/meminfo: memFree=15529396/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 518 0 0 0 0 0 0 0 25 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=47224 CPUtime=1.49 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 9363 0 0 0 148 1 0 0 20 0 1 0 515392996 48357376 7338 33554432000 4194304 4587354 140734037404528 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 11806 7338 326 96 0 8368 0
[pid=8444] ppid=8442 vsize=26500 CPUtime=1.49 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 5291 0 0 0 149 0 0 0 25 0 1 0 515392996 27136000 3945 33554432000 4194304 4785740 140733937810352 18446744073709551615 4553013 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 6625 3945 340 145 0 3670 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=1.18 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 117 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134530126 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=11704 CPUtime=1.48 cores=2
/proc/8446/stat : 8446 (lingeling) R 8442 8442 7235 0 -1 4202496 3951 0 0 0 148 0 0 0 20 0 1 0 515392996 11984896 2092 33554432000 4194304 4352236 140733805061792 18446744073709551615 4328100 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/8446/statm: 2926 2092 127 39 0 2006 0
[pid=8448] ppid=8442 vsize=25588 CPUtime=0.29 cores=6
/proc/8448/stat : 8448 (march_hi) R 8442 8442 7235 0 -1 4194304 8853 0 0 0 28 1 0 0 25 0 1 0 515393077 26202112 3930 33554432000 4194304 4331988 140737034749888 18446744073709551615 4252174 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8448/statm: 6397 3930 129 34 0 4839 0
Current children cumulated CPU time (s) 5.93
Current children cumulated vsize (KiB) 591236

[startup+3.10094 s]
/proc/loadavg: 5.18 6.69 8.17 6/177 8448
/proc/meminfo: memFree=15520840/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 518 0 0 0 0 0 0 0 18 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=84728 CPUtime=3.09 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 10390 0 0 0 308 1 0 0 25 0 1 0 515392996 86761472 8363 33554432000 4194304 4587354 140734037404528 18446744073709551615 4241101 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 21182 8363 326 96 0 17744 0
[pid=8444] ppid=8442 vsize=33744 CPUtime=3.09 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 7095 0 0 0 308 1 0 0 25 0 1 0 515392996 34553856 5749 33554432000 4194304 4785740 140733937810352 18446744073709551615 4309555 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 8436 5749 340 145 0 5481 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=1.97 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 196 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134530659 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=11704 CPUtime=3.09 cores=2
/proc/8446/stat : 8446 (lingeling) R 8442 8442 7235 0 -1 4202496 3951 0 0 0 308 1 0 0 25 0 1 0 515392996 11984896 2092 33554432000 4194304 4352236 140733805061792 18446744073709551615 4207805 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/8446/statm: 2926 2092 127 39 0 2006 0
[pid=8448] ppid=8442 vsize=59864 CPUtime=1.09 cores=6
/proc/8448/stat : 8448 (march_hi) R 8442 8442 7235 0 -1 4202496 27280 0 0 0 105 4 0 0 25 0 1 0 515393077 61300736 5408 33554432000 4194304 4331988 140737034749888 18446744073709551615 244446970986 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8448/statm: 14966 5408 151 34 0 13408 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 670260

[startup+6.30034 s]
/proc/loadavg: 5.18 6.69 8.17 6/177 8448
/proc/meminfo: memFree=15467900/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=84728 CPUtime=6.29 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 12277 0 0 0 627 2 0 0 25 0 1 0 515392996 86761472 10244 33554432000 4194304 4587354 140734037404528 18446744073709551615 4237665 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 21182 10244 326 96 0 17744 0
[pid=8444] ppid=8442 vsize=46492 CPUtime=6.28 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 10249 0 0 0 627 1 0 0 25 0 1 0 515392996 47607808 8903 33554432000 4194304 4785740 140733937810352 18446744073709551615 4304704 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 11623 8903 340 145 0 8668 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=3.56 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 355 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134528275 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=11704 CPUtime=6.29 cores=2
/proc/8446/stat : 8446 (lingeling) R 8442 8442 7235 0 -1 4202496 3951 0 0 0 628 1 0 0 25 0 1 0 515392996 11984896 2092 33554432000 4194304 4352236 140733805061792 18446744073709551615 4281083 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/8446/statm: 2926 2092 127 39 0 2006 0
[pid=8448] ppid=8442 vsize=57456 CPUtime=2.69 cores=6
/proc/8448/stat : 8448 (march_hi) R 8442 8442 7235 0 -1 4202496 34643 0 0 0 264 5 0 0 25 0 1 0 515393077 58834944 11649 33554432000 4194304 4331988 140737034749888 18446744073709551615 4228170 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8448/statm: 14364 11649 173 34 0 12806 0
Current children cumulated CPU time (s) 25.11
Current children cumulated vsize (KiB) 680600

[startup+12.7011 s]
/proc/loadavg: 5.22 6.65 8.14 6/177 8448
/proc/meminfo: memFree=15433320/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=133692 CPUtime=12.69 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 21388 0 0 0 1265 4 0 0 25 0 1 0 515392996 136900608 13904 33554432000 4194304 4587354 140734037404528 18446744073709551615 244446606021 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 33423 13906 337 96 0 29985 0
[pid=8444] ppid=8442 vsize=70096 CPUtime=12.69 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 16170 0 0 0 1266 3 0 0 25 0 1 0 515392996 71778304 14824 33554432000 4194304 4785740 140733937810352 18446744073709551615 4309555 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 17524 14824 340 145 0 14569 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=6.72 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 671 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134530207 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=16796 CPUtime=12.69 cores=2
/proc/8446/stat : 8446 (lingeling) R 8442 8442 7235 0 -1 4202496 5224 0 0 0 1268 1 0 0 25 0 1 0 515392996 17199104 3365 33554432000 4194304 4352236 140733805061792 18446744073709551615 4231316 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/8446/statm: 4199 3365 127 39 0 3279 0
[pid=8448] ppid=8442 vsize=62880 CPUtime=5.89 cores=6
/proc/8448/stat : 8448 (march_hi) R 8442 8442 7235 0 -1 4202496 35832 0 0 0 584 5 0 0 25 0 1 0 515393077 64389120 12838 33554432000 4194304 4331988 140737034749888 18446744073709551615 4229013 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8448/statm: 15720 12838 173 34 0 14162 0
Current children cumulated CPU time (s) 50.68
Current children cumulated vsize (KiB) 763684

[startup+25.5008 s]
/proc/loadavg: 5.26 6.61 8.12 6/177 8448
/proc/meminfo: memFree=15415148/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=59512 CPUtime=25.49 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 52179 0 0 0 2540 9 0 0 25 0 1 0 515392996 60940288 10890 33554432000 4194304 4587354 140734037404528 18446744073709551615 4238582 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 14878 10890 337 96 0 11440 0
[pid=8444] ppid=8442 vsize=81292 CPUtime=25.49 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 18927 0 0 0 2546 3 0 0 25 0 1 0 515392996 83243008 17581 33554432000 4194304 4785740 140733937810352 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 20323 17581 340 145 0 17368 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=13.1 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 1309 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134528511 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=17500 CPUtime=25.49 cores=2
/proc/8446/stat : 8446 (lingeling) R 8442 8442 7235 0 -1 4202496 5400 0 0 0 2548 1 0 0 25 0 1 0 515392996 17920000 3541 33554432000 4194304 4352236 140733805061792 18446744073709551615 4329876 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/8446/statm: 4375 3541 127 39 0 3455 0
[pid=8448] ppid=8442 vsize=64256 CPUtime=12.24 cores=6
/proc/8448/stat : 8448 (march_hi) R 8442 8442 7235 0 -1 4202496 36165 0 0 0 1218 6 0 0 25 0 1 0 515393077 65798144 13171 33554432000 4194304 4331988 140737034749888 18446744073709551615 4240214 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8448/statm: 16064 13171 173 34 0 14506 0
Current children cumulated CPU time (s) 101.81
Current children cumulated vsize (KiB) 702780

[startup+51.101 s]
/proc/loadavg: 5.16 6.46 8.01 6/177 8449
/proc/meminfo: memFree=15374432/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=122392 CPUtime=51.09 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 76904 0 0 0 5091 18 0 0 25 0 1 0 515392996 125329408 12472 33554432000 4194304 4587354 140734037404528 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 30598 12472 337 96 0 27160 0
[pid=8444] ppid=8442 vsize=108372 CPUtime=51.09 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 25588 0 0 0 5104 5 0 0 25 0 1 0 515392996 110972928 24242 33554432000 4194304 4785740 140733937810352 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 27093 24242 340 145 0 24138 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=25.82 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 2581 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134528913 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=23496 CPUtime=51.09 cores=2

################
# More data... #
################

/proc/8448/statm: 17318 14453 173 34 0 15760 0
Current children cumulated CPU time (s) 3764.15
Current children cumulated vsize (KiB) 2088612

[startup+1002.3 s]
/proc/loadavg: 5.07 5.12 6.11 6/177 8484
/proc/meminfo: memFree=15078356/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=1133680 CPUtime=1002.31 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 590770 0 0 0 100011 220 0 0 25 0 1 0 515392996 1160888320 26922 33554432000 4194304 4587354 140734037404528 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 283420 26922 337 96 0 279982 0
[pid=8444] ppid=8442 vsize=340052 CPUtime=1002.32 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 82705 0 0 0 100212 20 0 0 25 0 1 0 515392996 348213248 81359 33554432000 4194304 4785740 140733937810352 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 85013 81359 340 145 0 82058 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=498.87 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 49886 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134528661 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=27588 CPUtime=1002.32 cores=2
/proc/8446/stat : 8446 (lingeling) R 8442 8442 7235 0 -1 4202496 7922 0 0 0 100230 2 0 0 25 0 1 0 515392996 28250112 6063 33554432000 4194304 4352236 140733805061792 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/8446/statm: 6897 6063 127 39 0 5977 0
[pid=8448] ppid=8442 vsize=69404 CPUtime=498.02 cores=6
/proc/8448/stat : 8448 (march_hi) R 8442 8442 7235 0 -1 4202496 37469 0 0 0 49796 6 0 0 25 0 1 0 515393077 71069696 14475 33554432000 4194304 4331988 140737034749888 18446744073709551615 4288315 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8448/statm: 17351 14475 173 34 0 15793 0
Current children cumulated CPU time (s) 4003.84
Current children cumulated vsize (KiB) 2050944

[startup+1062.3 s]
/proc/loadavg: 5.06 5.11 6.05 6/177 8485
/proc/meminfo: memFree=15097940/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=1147388 CPUtime=1062.31 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 626254 0 0 0 105995 236 0 0 25 0 1 0 515392996 1174925312 21722 33554432000 4194304 4587354 140734037404528 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 286847 21722 337 96 0 283409 0
[pid=8444] ppid=8442 vsize=340052 CPUtime=1062.32 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 82722 0 0 0 106212 20 0 0 25 0 1 0 515392996 348213248 81376 33554432000 4194304 4785740 140733937810352 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 85013 81376 340 145 0 82058 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=528.72 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 52871 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=29636 CPUtime=1062.32 cores=2
/proc/8446/stat : 8446 (lingeling) R 8442 8442 7235 0 -1 4202496 8434 0 0 0 106230 2 0 0 25 0 1 0 515392996 30347264 6575 33554432000 4194304 4352236 140733805061792 18446744073709551615 4328991 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/8446/statm: 7409 6575 127 39 0 6489 0
[pid=8448] ppid=8442 vsize=69536 CPUtime=527.84 cores=6
/proc/8448/stat : 8448 (march_hi) R 8442 8442 7235 0 -1 4202496 37492 0 0 0 52778 6 0 0 25 0 1 0 515393077 71204864 14498 33554432000 4194304 4331988 140737034749888 18446744073709551615 4284556 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8448/statm: 17384 14498 173 34 0 15826 0
Current children cumulated CPU time (s) 4243.51
Current children cumulated vsize (KiB) 2066832

[startup+1122.3 s]
/proc/loadavg: 5.13 5.12 5.99 6/177 8490
/proc/meminfo: memFree=15120988/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=1140384 CPUtime=1122.32 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 652864 0 0 0 111980 252 0 0 25 0 1 0 515392996 1167753216 15744 33554432000 4194304 4587354 140734037404528 18446744073709551615 244446641931 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 285096 15744 337 96 0 281658 0
[pid=8444] ppid=8442 vsize=340052 CPUtime=1122.32 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 82736 0 0 0 112212 20 0 0 25 0 1 0 515392996 348213248 81390 33554432000 4194304 4785740 140733937810352 18446744073709551615 4550516 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 85013 81390 340 145 0 82058 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=558.53 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 55852 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=29636 CPUtime=1122.33 cores=2
/proc/8446/stat : 8446 (lingeling) R 8442 8442 7235 0 -1 4202496 8434 0 0 0 112231 2 0 0 25 0 1 0 515392996 30347264 6575 33554432000 4194304 4352236 140733805061792 18446744073709551615 4207874 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/8446/statm: 7409 6575 127 39 0 6489 0
[pid=8448] ppid=8442 vsize=69536 CPUtime=557.71 cores=6
/proc/8448/stat : 8448 (march_hi) R 8442 8442 7235 0 -1 4202496 37513 0 0 0 55765 6 0 0 25 0 1 0 515393077 71204864 14519 33554432000 4194304 4331988 140737034749888 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8448/statm: 17384 14519 173 34 0 15826 0
Current children cumulated CPU time (s) 4483.21
Current children cumulated vsize (KiB) 2059828

[startup+1182.3 s]
/proc/loadavg: 5.16 5.13 5.93 6/177 8491
/proc/meminfo: memFree=15087196/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=1170900 CPUtime=1182.32 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 678219 0 0 0 117967 265 0 0 25 0 1 0 515392996 1199001600 24342 33554432000 4194304 4587354 140734037404528 18446744073709551615 4238600 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 292725 24342 337 96 0 289287 0
[pid=8444] ppid=8442 vsize=340052 CPUtime=1182.32 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 82762 0 0 0 118212 20 0 0 25 0 1 0 515392996 348213248 81416 33554432000 4194304 4785740 140733937810352 18446744073709551615 4278600 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 85013 81416 340 145 0 82058 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=588.41 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 58840 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134528511 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=29636 CPUtime=1182.33 cores=2
/proc/8446/stat : 8446 (lingeling) R 8442 8442 7235 0 -1 4202496 8434 0 0 0 118231 2 0 0 25 0 1 0 515392996 30347264 6575 33554432000 4194304 4352236 140733805061792 18446744073709551615 4328390 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/8446/statm: 7409 6575 127 39 0 6489 0
[pid=8448] ppid=8442 vsize=69668 CPUtime=587.5 cores=6
/proc/8448/stat : 8448 (march_hi) R 8442 8442 7235 0 -1 4202496 37534 0 0 0 58744 6 0 0 25 0 1 0 515393077 71340032 14540 33554432000 4194304 4331988 140737034749888 18446744073709551615 4297344 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8448/statm: 17417 14540 173 34 0 15859 0
Current children cumulated CPU time (s) 4722.88
Current children cumulated vsize (KiB) 2090476



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 5.18 5.13 5.92 6/177 8492
/proc/meminfo: memFree=15115712/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=1159284 CPUtime=1201.72 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 687392 0 0 0 119903 269 0 0 25 0 1 0 515392996 1187106816 16572 33554432000 4194304 4587354 140734037404528 18446744073709551615 4236384 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 289821 16572 337 96 0 286383 0
[pid=8444] ppid=8442 vsize=341508 CPUtime=1201.74 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 83103 0 0 0 120153 21 0 0 25 0 1 0 515392996 349704192 81757 33554432000 4194304 4785740 140733937810352 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 85377 81757 340 145 0 82422 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=598.01 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 59800 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=29636 CPUtime=1201.73 cores=2
/proc/8446/stat : 8446 (lingeling) R 8442 8442 7235 0 -1 4202496 8434 0 0 0 120171 2 0 0 25 0 1 0 515392996 30347264 6575 33554432000 4194304 4352236 140733805061792 18446744073709551615 4328022 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/8446/statm: 7409 6575 127 39 0 6489 0
[pid=8448] ppid=8442 vsize=69668 CPUtime=597.2 cores=6
/proc/8448/stat : 8448 (march_hi) R 8442 8442 7235 0 -1 4202496 37540 0 0 0 59714 6 0 0 25 0 1 0 515393077 71340032 14546 33554432000 4194304 4331988 140737034749888 18446744073709551615 4237171 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8448/statm: 17417 14546 173 34 0 15859 0
Current children cumulated CPU time (s) 4800.4
Current children cumulated vsize (KiB) 2080316

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 5.18 5.13 5.92 6/177 8492
/proc/meminfo: memFree=15115712/32951124 swapFree=67111528/67111528
[pid=8442] ppid=8440 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/8442/stat : 8442 (ppfolio) S 8440 8442 7235 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 515392996 18763776 389 33554432000 4194304 4287625 140737073975152 18446744073709551615 244446965263 0 0 4096 65536 3279553743514763263 0 0 17 2 0 0 0
/proc/8442/statm: 4581 389 341 23 0 91 0
[pid=8443] ppid=8442 vsize=1159284 CPUtime=1201.72 cores=0
/proc/8443/stat : 8443 (cryptominisat) R 8442 8442 7235 0 -1 4202496 687392 0 0 0 119903 269 0 0 25 0 1 0 515392996 1187106816 16572 33554432000 4194304 4587354 140734037404528 18446744073709551615 4236384 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8443/statm: 289821 16572 337 96 0 286383 0
[pid=8444] ppid=8442 vsize=341508 CPUtime=1201.74 cores=4
/proc/8444/stat : 8444 (clasp) R 8442 8442 7235 0 -1 4202496 83103 0 0 0 120153 21 0 0 25 0 1 0 515392996 349704192 81757 33554432000 4194304 4785740 140733937810352 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/8444/statm: 85377 81757 340 145 0 82422 0
[pid=8445] ppid=8442 vsize=461896 CPUtime=598.01 cores=6
/proc/8445/stat : 8445 (TNM) R 8442 8442 7235 0 -1 4202496 4293 0 0 0 59800 1 0 0 25 0 1 0 515392996 472981504 4190 33554432000 134512640 134534564 4294887744 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8445/statm: 115474 4190 95 6 0 114140 0
[pid=8446] ppid=8442 vsize=29636 CPUtime=1201.73 cores=2
/proc/8446/stat : 8446 (lingeling) R 8442 8442 7235 0 -1 4202496 8434 0 0 0 120171 2 0 0 25 0 1 0 515392996 30347264 6575 33554432000 4194304 4352236 140733805061792 18446744073709551615 4328022 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/8446/statm: 7409 6575 127 39 0 6489 0
[pid=8448] ppid=8442 vsize=69668 CPUtime=597.2 cores=6
/proc/8448/stat : 8448 (march_hi) R 8442 8442 7235 0 -1 4202496 37540 0 0 0 59714 6 0 0 25 0 1 0 515393077 71340032 14546 33554432000 4194304 4331988 140737034749888 18446744073709551615 4237171 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/8448/statm: 17417 14546 173 34 0 15859 0
Current children cumulated CPU time (s) 4800.4
Current children cumulated vsize (KiB) 2080316

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 8442 and gives
#  childrusage.ru_utime.tv_sec=2997
#  childrusage.ru_utime.tv_usec=899250
#  childrusage.ru_stime.tv_sec=2
#  childrusage.ru_stime.tv_usec=801574
# CPU time returned by wait4() is 3000.7
# while last known CPU time is 4800.4
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.4
CPU user time (s): 4797.41
CPU system time (s): 2.99
CPU usage (%): 399.463
Max. virtual memory (cumulated for all children) (KiB): 3141116

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2997.9
system time used= 2.80157
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 733887
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 282
involuntary context switches= 16787

runsolver used 2.17267 second user time and 4.34834 second system time

The end

Launcher Data 

Begin job on node105 at 2011-04-29 02:45:26
IDJOB=3190845
IDBENCH=82576
IDSOLVER=1612
FILE ID=node105/3190845-1304037925
RUNJOBID= node105-1304028688-7254
PBS_JOBID= 13170347
Free space on /tmp= 71936 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v25000-c105000-S109959541-056-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3190845-1304037925/watcher-3190845-1304037925 -o /tmp/evaluation-result-3190845-1304037925/solver-3190845-1304037925 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3190845-1304037925.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5f909806405bfbe25b7db5afcabe8fc9
RANDOM SEED=2074835380

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15608112 kB
Buffers:       2299512 kB
Cached:       14407860 kB
SwapCached:          0 kB
Active:        8901208 kB
Inactive:      7872340 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15608112 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           12076 kB
Writeback:           0 kB
AnonPages:       65412 kB
Mapped:          14716 kB
Slab:           506948 kB
PageTables:       4436 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   188808 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71928 MiB
End job on node105 at 2011-04-29 03:05:29