Trace number 3190839

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.21 1201.71

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v40000-c168000-S709305826-098-UNKNOWN.cnf
MD5SUM83f9a63a559155a858e739be2c598060
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark45.6951
Satisfiable
(Un)Satisfiability was proved
Number of variables40000
Number of clauses168000
Sum of the clauses size504000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3168000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 4
0.00/0.00	Starting TNM 2009 on core(s) 6
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 6

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3190839-1304037624/watcher-3190839-1304037624 -o /tmp/evaluation-result-3190839-1304037624/solver-3190839-1304037624 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3190839-1304037624.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.74 10.06 10.07 7/182 23657
/proc/meminfo: memFree=26296620/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 520 0 0 0 0 0 0 0 19 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0

[startup+1.23133 s]
/proc/loadavg: 9.74 10.06 10.07 7/182 23657
/proc/meminfo: memFree=26296620/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 520 0 0 0 0 0 0 0 19 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.33131 s]
/proc/loadavg: 9.74 10.06 10.07 7/182 23657
/proc/meminfo: memFree=26296620/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 520 0 0 0 0 0 0 0 19 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.4003 s]
/proc/loadavg: 9.74 10.06 10.07 7/182 23657
/proc/meminfo: memFree=26296620/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 520 0 0 0 0 0 0 0 19 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.53127 s]
/proc/loadavg: 9.74 10.06 10.07 7/182 23657
/proc/meminfo: memFree=26296620/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 520 0 0 0 0 0 0 0 19 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.60026 s]
/proc/loadavg: 9.74 10.06 10.07 7/182 23657
/proc/meminfo: memFree=26296620/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 520 0 0 0 0 0 0 0 19 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+3.101 s]
/proc/loadavg: 9.74 10.06 10.07 12/188 23663
/proc/meminfo: memFree=26173304/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 520 0 0 0 0 0 0 0 18 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=97824 CPUtime=3.09 cores=0
/proc/23658/stat : 23658 (cryptominisat) R 23657 23657 22773 0 -1 4202496 14755 0 0 0 306 3 0 0 25 0 1 0 515360400 100171776 11571 33554432000 4194304 4587354 140735689724272 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23658/statm: 24456 11571 326 96 0 21018 0
[pid=23659] ppid=23657 vsize=35608 CPUtime=3.09 cores=4
/proc/23659/stat : 23659 (clasp) R 23657 23657 22773 0 -1 4202496 8269 0 0 0 308 1 0 0 25 0 1 0 515360400 36462592 6182 33554432000 4194304 4785740 140734226892656 18446744073709551615 4303721 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/23659/statm: 8902 6182 340 145 0 5947 0
[pid=23660] ppid=23657 vsize=467968 CPUtime=1.95 cores=6
/proc/23660/stat : 23660 (TNM) R 23657 23657 22773 0 -1 4202496 6719 0 0 0 195 0 0 0 21 0 1 0 515360400 479199232 6616 33554432000 134512640 134534564 4288815008 18446744073709551615 134525956 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23660/statm: 116992 6616 95 6 0 115658 0
[pid=23661] ppid=23657 vsize=19404 CPUtime=3.09 cores=2
/proc/23661/stat : 23661 (lingeling) R 23657 23657 22773 0 -1 4202496 6668 0 0 0 308 1 0 0 25 0 1 0 515360400 19869696 3506 33554432000 4194304 4352236 140736914863824 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/23661/statm: 4851 3506 127 39 0 3931 0
[pid=23662] ppid=23657 vsize=36980 CPUtime=1.11 cores=6
/proc/23662/stat : 23662 (march_hi) R 23657 23657 22773 0 -1 4202496 25377 0 0 0 109 2 0 0 25 0 1 0 515360400 37867520 6173 33554432000 4194304 4331988 140736142641504 18446744073709551615 4252197 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23662/statm: 9245 6173 141 34 0 7687 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 676108

[startup+6.30044 s]
/proc/loadavg: 9.76 10.06 10.07 12/188 23663
/proc/meminfo: memFree=26121676/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=97824 CPUtime=6.28 cores=0
/proc/23658/stat : 23658 (cryptominisat) R 23657 23657 22773 0 -1 4202496 16462 0 0 0 625 3 0 0 25 0 1 0 515360400 100171776 13272 33554432000 4194304 4587354 140735689724272 18446744073709551615 4238739 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23658/statm: 24456 13272 326 96 0 21018 0
[pid=23659] ppid=23657 vsize=47064 CPUtime=6.29 cores=4
/proc/23659/stat : 23659 (clasp) R 23657 23657 22773 0 -1 4202496 11126 0 0 0 628 1 0 0 25 0 1 0 515360400 48193536 9039 33554432000 4194304 4785740 140734226892656 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/23659/statm: 11766 9039 340 145 0 8811 0
[pid=23660] ppid=23657 vsize=467968 CPUtime=3.53 cores=6
/proc/23660/stat : 23660 (TNM) R 23657 23657 22773 0 -1 4202496 6719 0 0 0 353 0 0 0 25 0 1 0 515360400 479199232 6616 33554432000 134512640 134534564 4288815008 18446744073709551615 134528523 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23660/statm: 116992 6616 95 6 0 115658 0
[pid=23661] ppid=23657 vsize=19404 CPUtime=6.29 cores=2
/proc/23661/stat : 23661 (lingeling) R 23657 23657 22773 0 -1 4202496 6668 0 0 0 628 1 0 0 25 0 1 0 515360400 19869696 3506 33554432000 4194304 4352236 140736914863824 18446744073709551615 4207874 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/23661/statm: 4851 3506 127 39 0 3931 0
[pid=23662] ppid=23657 vsize=86296 CPUtime=2.71 cores=6
/proc/23662/stat : 23662 (march_hi) R 23657 23657 22773 0 -1 4202496 52577 0 0 0 265 6 0 0 25 0 1 0 515360400 88367104 15249 33554432000 4194304 4331988 140736142641504 18446744073709551615 4263203 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23662/statm: 21574 15249 173 34 0 20016 0
Current children cumulated CPU time (s) 25.1
Current children cumulated vsize (KiB) 736880

[startup+12.7003 s]
/proc/loadavg: 9.78 10.06 10.06 11/188 23663
/proc/meminfo: memFree=26039028/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=69628 CPUtime=12.69 cores=0
/proc/23658/stat : 23658 (cryptominisat) R 23657 23657 22773 0 -1 4202496 21646 0 0 0 1263 6 0 0 25 0 1 0 515360400 71299072 11319 33554432000 4194304 4587354 140735689724272 18446744073709551615 4241265 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23658/statm: 17407 11319 327 96 0 13969 0
[pid=23659] ppid=23657 vsize=67976 CPUtime=12.69 cores=4
/proc/23659/stat : 23659 (clasp) R 23657 23657 22773 0 -1 4202496 16332 0 0 0 1266 3 0 0 25 0 1 0 515360400 69607424 14245 33554432000 4194304 4785740 140734226892656 18446744073709551615 4552970 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/23659/statm: 16994 14245 340 145 0 14039 0
[pid=23660] ppid=23657 vsize=467968 CPUtime=6.69 cores=6
/proc/23660/stat : 23660 (TNM) R 23657 23657 22773 0 -1 4202496 6719 0 0 0 669 0 0 0 25 0 1 0 515360400 479199232 6616 33554432000 134512640 134534564 4288815008 18446744073709551615 134528737 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23660/statm: 116992 6616 95 6 0 115658 0
[pid=23661] ppid=23657 vsize=24756 CPUtime=12.69 cores=2
/proc/23661/stat : 23661 (lingeling) R 23657 23657 22773 0 -1 4202496 8517 0 0 0 1268 1 0 0 25 0 1 0 515360400 25350144 5355 33554432000 4194304 4352236 140736914863824 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/23661/statm: 6189 5355 127 39 0 5269 0
[pid=23662] ppid=23657 vsize=88116 CPUtime=5.91 cores=6
/proc/23662/stat : 23662 (march_hi) R 23657 23657 22773 0 -1 4202496 55800 0 0 0 585 6 0 0 25 0 1 0 515360400 90230784 18472 33554432000 4194304 4331988 140736142641504 18446744073709551615 4265440 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23662/statm: 22029 18472 173 34 0 20471 0
Current children cumulated CPU time (s) 50.67
Current children cumulated vsize (KiB) 736768

[startup+25.5011 s]
/proc/loadavg: 9.83 10.05 10.06 12/188 23664
/proc/meminfo: memFree=25952348/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=158592 CPUtime=25.49 cores=0
/proc/23658/stat : 23658 (cryptominisat) R 23657 23657 22773 0 -1 4202496 50537 0 0 0 2537 12 0 0 25 0 1 0 515360400 162398208 19502 33554432000 4194304 4587354 140735689724272 18446744073709551615 4236957 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23658/statm: 39648 19502 337 96 0 36210 0
[pid=23659] ppid=23657 vsize=106704 CPUtime=25.49 cores=4
/proc/23659/stat : 23659 (clasp) R 23657 23657 22773 0 -1 4202496 25941 0 0 0 2543 6 0 0 25 0 1 0 515360400 109264896 23854 33554432000 4194304 4785740 140734226892656 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/23659/statm: 26676 23854 340 145 0 23721 0
[pid=23660] ppid=23657 vsize=467968 CPUtime=13.09 cores=6
/proc/23660/stat : 23660 (TNM) R 23657 23657 22773 0 -1 4202496 6719 0 0 0 1309 0 0 0 25 0 1 0 515360400 479199232 6616 33554432000 134512640 134534564 4288815008 18446744073709551615 134529031 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23660/statm: 116992 6616 95 6 0 115658 0
[pid=23661] ppid=23657 vsize=24756 CPUtime=25.49 cores=2
/proc/23661/stat : 23661 (lingeling) R 23657 23657 22773 0 -1 4202496 8517 0 0 0 2548 1 0 0 25 0 1 0 515360400 25350144 5355 33554432000 4194304 4352236 140736914863824 18446744073709551615 4278833 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/23661/statm: 6189 5355 127 39 0 5269 0
[pid=23662] ppid=23657 vsize=96728 CPUtime=12.25 cores=6
/proc/23662/stat : 23662 (march_hi) R 23657 23657 22773 0 -1 4202496 57513 0 0 0 1218 7 0 0 25 0 1 0 515360400 99049472 20185 33554432000 4194304 4331988 140736142641504 18446744073709551615 4265510 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23662/statm: 24182 20185 173 34 0 22624 0
Current children cumulated CPU time (s) 101.81
Current children cumulated vsize (KiB) 873072

[startup+51.1006 s]
/proc/loadavg: 10.01 10.08 10.07 12/188 23664
/proc/meminfo: memFree=25898628/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=144392 CPUtime=51.09 cores=0
/proc/23658/stat : 23658 (cryptominisat) R 23657 23657 22773 0 -1 4202496 88003 0 0 0 5087 22 0 0 25 0 1 0 515360400 147857408 30917 33554432000 4194304 4587354 140735689724272 18446744073709551615 4372590 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23658/statm: 36098 30917 337 96 0 32660 0
[pid=23659] ppid=23657 vsize=146928 CPUtime=51.08 cores=4
/proc/23659/stat : 23659 (clasp) R 23657 23657 22773 0 -1 4202496 35956 0 0 0 5099 9 0 0 25 0 1 0 515360400 150454272 33869 33554432000 4194304 4785740 140734226892656 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/23659/statm: 36732 33869 340 145 0 33777 0
[pid=23660] ppid=23657 vsize=467968 CPUtime=25.79 cores=6
/proc/23660/stat : 23660 (TNM) R 23657 23657 22773 0 -1 4202496 6719 0 0 0 2579 0 0 0 25 0 1 0 515360400 479199232 6616 33554432000 134512640 134534564 4288815008 18446744073709551615 134528504 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23660/statm: 116992 6616 95 6 0 115658 0
[pid=23661] ppid=23657 vsize=27408 CPUtime=51.09 cores=2
/proc/23661/stat : 23661 (lingeling) R 23657 23657 22773 0 -1 4202496 9180 0 0 0 5108 1 0 0 25 0 1 0 515360400 28065792 6018 33554432000 4194304 4352236 140736914863824 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/23661/statm: 6852 6018 127 39 0 5932 0
[pid=23662] ppid=23657 vsize=99500 CPUtime=25.01 cores=6
/proc/23662/stat : 23662 (march_hi) R 23657 23657 22773 0 -1 4202496 58225 0 0 0 2493 8 0 0 25 0 1 0 515360400 101888000 20897 33554432000 4194304 4331988 140736142641504 18446744073709551615 4263127 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23662/statm: 24875 20897 173 34 0 23317 0
Current children cumulated CPU time (s) 204.06
Current children cumulated vsize (KiB) 904520

[startup+102.301 s]
/proc/loadavg: 10.23 10.12 10.08 12/188 23666
/proc/meminfo: memFree=25875592/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=1987288 CPUtime=102.29 cores=0

################
# More data... #
################

/proc/23662/statm: 26261 22305 173 34 0 24703 0
Current children cumulated CPU time (s) 3763.99
Current children cumulated vsize (KiB) 3033500

[startup+1002.3 s]
/proc/loadavg: 10.24 10.16 10.10 12/188 23690
/proc/meminfo: memFree=25458200/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=1980552 CPUtime=1002.32 cores=0
/proc/23658/stat : 23658 (cryptominisat) R 23657 23657 22773 0 -1 4202496 639379 0 0 0 100008 224 0 0 25 0 1 0 515360400 2028085248 26705 33554432000 4194304 4587354 140735689724272 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23658/statm: 495138 26705 337 96 0 491700 0
[pid=23659] ppid=23657 vsize=449784 CPUtime=1002.21 cores=4
/proc/23659/stat : 23659 (clasp) R 23657 23657 22773 0 -1 4202496 110110 0 0 0 100190 31 0 0 25 0 1 0 515360400 460578816 108023 33554432000 4194304 4785740 140734226892656 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/23659/statm: 112446 108023 340 145 0 109491 0
[pid=23660] ppid=23657 vsize=467968 CPUtime=498.81 cores=6
/proc/23660/stat : 23660 (TNM) R 23657 23657 22773 0 -1 4202496 6719 0 0 0 49881 0 0 0 25 0 1 0 515360400 479199232 6616 33554432000 134512640 134534564 4288815008 18446744073709551615 134530890 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23660/statm: 116992 6616 95 6 0 115658 0
[pid=23661] ppid=23657 vsize=30468 CPUtime=1002.3 cores=2
/proc/23661/stat : 23661 (lingeling) R 23657 23657 22773 0 -1 4202496 9945 0 0 0 100229 1 0 0 25 0 1 0 515360400 31199232 6783 33554432000 4194304 4352236 140736914863824 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/23661/statm: 7617 6783 127 39 0 6697 0
[pid=23662] ppid=23657 vsize=105176 CPUtime=498.02 cores=6
/proc/23662/stat : 23662 (march_hi) R 23657 23657 22773 0 -1 4202496 59662 0 0 0 49793 9 0 0 25 0 1 0 515360400 107700224 22334 33554432000 4194304 4331988 140736142641504 18446744073709551615 4226474 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23662/statm: 26294 22334 173 34 0 24736 0
Current children cumulated CPU time (s) 4003.66
Current children cumulated vsize (KiB) 3052272

[startup+1062.3 s]
/proc/loadavg: 10.21 10.16 10.10 12/188 23691
/proc/meminfo: memFree=25506168/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=168252 CPUtime=1062.31 cores=0
/proc/23658/stat : 23658 (cryptominisat) R 23657 23657 22773 0 -1 4202496 657390 0 0 0 105998 233 0 0 25 0 1 0 515360400 172290048 19927 33554432000 4194304 4587354 140735689724272 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23658/statm: 42063 19927 337 96 0 38625 0
[pid=23659] ppid=23657 vsize=449784 CPUtime=1062.21 cores=4
/proc/23659/stat : 23659 (clasp) R 23657 23657 22773 0 -1 4202496 110147 0 0 0 106190 31 0 0 25 0 1 0 515360400 460578816 108060 33554432000 4194304 4785740 140734226892656 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/23659/statm: 112446 108060 340 145 0 109491 0
[pid=23660] ppid=23657 vsize=467968 CPUtime=528.67 cores=6
/proc/23660/stat : 23660 (TNM) R 23657 23657 22773 0 -1 4202496 6719 0 0 0 52867 0 0 0 25 0 1 0 515360400 479199232 6616 33554432000 134512640 134534564 4288815008 18446744073709551615 134528661 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23660/statm: 116992 6616 95 6 0 115658 0
[pid=23661] ppid=23657 vsize=32516 CPUtime=1062.3 cores=2
/proc/23661/stat : 23661 (lingeling) R 23657 23657 22773 0 -1 4202496 10457 0 0 0 106229 1 0 0 25 0 1 0 515360400 33296384 7295 33554432000 4194304 4352236 140736914863824 18446744073709551615 4276751 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/23661/statm: 8129 7295 127 39 0 7209 0
[pid=23662] ppid=23657 vsize=105308 CPUtime=527.83 cores=6
/proc/23662/stat : 23662 (march_hi) R 23657 23657 22773 0 -1 4202496 59696 0 0 0 52774 9 0 0 25 0 1 0 515360400 107835392 22368 33554432000 4194304 4331988 140736142641504 18446744073709551615 4229026 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23662/statm: 26327 22368 173 34 0 24769 0
Current children cumulated CPU time (s) 4243.32
Current children cumulated vsize (KiB) 1242152

[startup+1122.3 s]
/proc/loadavg: 9.79 10.08 10.07 6/177 23711
/proc/meminfo: memFree=25640568/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=2018424 CPUtime=1122.31 cores=0
/proc/23658/stat : 23658 (cryptominisat) R 23657 23657 22773 0 -1 4202496 677999 0 0 0 111988 243 0 0 25 0 1 0 515360400 2066866176 27664 33554432000 4194304 4587354 140735689724272 18446744073709551615 4238593 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23658/statm: 504606 27664 337 96 0 501168 0
[pid=23659] ppid=23657 vsize=519488 CPUtime=1122.2 cores=4
/proc/23659/stat : 23659 (clasp) R 23657 23657 22773 0 -1 4202496 126638 0 0 0 112186 34 0 0 25 0 1 0 515360400 531955712 124551 33554432000 4194304 4785740 140734226892656 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/23659/statm: 129872 124551 340 145 0 126917 0
[pid=23660] ppid=23657 vsize=467968 CPUtime=558.5 cores=6
/proc/23660/stat : 23660 (TNM) R 23657 23657 22773 0 -1 4202496 6719 0 0 0 55850 0 0 0 25 0 1 0 515360400 479199232 6616 33554432000 134512640 134534564 4288815008 18446744073709551615 134528534 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23660/statm: 116992 6616 95 6 0 115658 0
[pid=23661] ppid=23657 vsize=32516 CPUtime=1122.32 cores=2
/proc/23661/stat : 23661 (lingeling) R 23657 23657 22773 0 -1 4202496 10457 0 0 0 112230 2 0 0 25 0 1 0 515360400 33296384 7295 33554432000 4194304 4352236 140736914863824 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/23661/statm: 8129 7295 127 39 0 7209 0
[pid=23662] ppid=23657 vsize=105440 CPUtime=557.68 cores=6
/proc/23662/stat : 23662 (march_hi) R 23657 23657 22773 0 -1 4202496 59727 0 0 0 55759 9 0 0 25 0 1 0 515360400 107970560 22399 33554432000 4194304 4331988 140736142641504 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23662/statm: 26360 22399 173 34 0 24802 0
Current children cumulated CPU time (s) 4483.01
Current children cumulated vsize (KiB) 3162160

[startup+1182.3 s]
/proc/loadavg: 6.82 9.17 9.76 6/177 23713
/proc/meminfo: memFree=25671328/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=1978728 CPUtime=1182.31 cores=0
/proc/23658/stat : 23658 (cryptominisat) R 23657 23657 22773 0 -1 4202496 696455 0 0 0 117979 252 0 0 25 0 1 0 515360400 2026217472 19788 33554432000 4194304 4587354 140735689724272 18446744073709551615 4243788 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23658/statm: 494682 19788 337 96 0 491244 0
[pid=23659] ppid=23657 vsize=519488 CPUtime=1182.2 cores=4
/proc/23659/stat : 23659 (clasp) R 23657 23657 22773 0 -1 4202496 126641 0 0 0 118186 34 0 0 25 0 1 0 515360400 531955712 124554 33554432000 4194304 4785740 140734226892656 18446744073709551615 4278587 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/23659/statm: 129872 124554 340 145 0 126917 0
[pid=23660] ppid=23657 vsize=467968 CPUtime=588.33 cores=6
/proc/23660/stat : 23660 (TNM) R 23657 23657 22773 0 -1 4202496 6719 0 0 0 58833 0 0 0 25 0 1 0 515360400 479199232 6616 33554432000 134512640 134534564 4288815008 18446744073709551615 134530126 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23660/statm: 116992 6616 95 6 0 115658 0
[pid=23661] ppid=23657 vsize=32516 CPUtime=1182.31 cores=2
/proc/23661/stat : 23661 (lingeling) R 23657 23657 22773 0 -1 4202496 10457 0 0 0 118229 2 0 0 25 0 1 0 515360400 33296384 7295 33554432000 4194304 4352236 140736914863824 18446744073709551615 4329086 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/23661/statm: 8129 7295 127 39 0 7209 0
[pid=23662] ppid=23657 vsize=105572 CPUtime=587.52 cores=6
/proc/23662/stat : 23662 (march_hi) R 23657 23657 22773 0 -1 4202496 59760 0 0 0 58743 9 0 0 25 0 1 0 515360400 108105728 22432 33554432000 4194304 4331988 140736142641504 18446744073709551615 4228963 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23662/statm: 26393 22432 173 34 0 24835 0
Current children cumulated CPU time (s) 4722.67
Current children cumulated vsize (KiB) 3122596



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 6.30 8.90 9.65 6/177 23713
/proc/meminfo: memFree=25637708/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=1978728 CPUtime=1201.73 cores=0
/proc/23658/stat : 23658 (cryptominisat) R 23657 23657 22773 0 -1 4202496 701949 0 0 0 119917 256 0 0 25 0 1 0 515360400 2026217472 25265 33554432000 4194304 4587354 140735689724272 18446744073709551615 4241080 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23658/statm: 494682 25265 337 96 0 491244 0
[pid=23659] ppid=23657 vsize=531880 CPUtime=1201.61 cores=4
/proc/23659/stat : 23659 (clasp) R 23657 23657 22773 0 -1 4202496 129732 0 0 0 120126 35 0 0 25 0 1 0 515360400 544645120 127645 33554432000 4194304 4785740 140734226892656 18446744073709551615 4287194 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/23659/statm: 132970 127645 340 145 0 130015 0
[pid=23660] ppid=23657 vsize=467968 CPUtime=598 cores=6
/proc/23660/stat : 23660 (TNM) R 23657 23657 22773 0 -1 4202496 6719 0 0 0 59800 0 0 0 25 0 1 0 515360400 479199232 6616 33554432000 134512640 134534564 4288815008 18446744073709551615 134530659 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23660/statm: 116992 6616 95 6 0 115658 0
[pid=23661] ppid=23657 vsize=32516 CPUtime=1201.72 cores=2
/proc/23661/stat : 23661 (lingeling) R 23657 23657 22773 0 -1 4202496 10457 0 0 0 120170 2 0 0 25 0 1 0 515360400 33296384 7295 33554432000 4194304 4352236 140736914863824 18446744073709551615 4231116 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/23661/statm: 8129 7295 127 39 0 7209 0
[pid=23662] ppid=23657 vsize=105572 CPUtime=597.15 cores=6
/proc/23662/stat : 23662 (march_hi) R 23657 23657 22773 0 -1 4202496 59768 0 0 0 59706 9 0 0 25 0 1 0 515360400 108105728 22440 33554432000 4194304 4331988 140736142641504 18446744073709551615 4292076 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23662/statm: 26393 22440 173 34 0 24835 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 3134988

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 6.30 8.90 9.65 6/177 23713
/proc/meminfo: memFree=25637708/32951124 swapFree=67111364/67111528
[pid=23657] ppid=23655 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/23657/stat : 23657 (ppfolio) S 23655 23657 22773 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 515360399 18763776 389 33554432000 4194304 4287625 140737042919824 18446744073709551615 228244368911 0 0 4096 65536 18248611945240330239 0 0 17 4 0 0 0
/proc/23657/statm: 4581 389 341 23 0 91 0
[pid=23658] ppid=23657 vsize=1978728 CPUtime=1201.73 cores=0
/proc/23658/stat : 23658 (cryptominisat) R 23657 23657 22773 0 -1 4202496 701949 0 0 0 119917 256 0 0 25 0 1 0 515360400 2026217472 25265 33554432000 4194304 4587354 140735689724272 18446744073709551615 4241080 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23658/statm: 494682 25265 337 96 0 491244 0
[pid=23659] ppid=23657 vsize=531880 CPUtime=1201.61 cores=4
/proc/23659/stat : 23659 (clasp) R 23657 23657 22773 0 -1 4202496 129732 0 0 0 120126 35 0 0 25 0 1 0 515360400 544645120 127645 33554432000 4194304 4785740 140734226892656 18446744073709551615 4287194 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/23659/statm: 132970 127645 340 145 0 130015 0
[pid=23660] ppid=23657 vsize=467968 CPUtime=598 cores=6
/proc/23660/stat : 23660 (TNM) R 23657 23657 22773 0 -1 4202496 6719 0 0 0 59800 0 0 0 25 0 1 0 515360400 479199232 6616 33554432000 134512640 134534564 4288815008 18446744073709551615 134530659 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23660/statm: 116992 6616 95 6 0 115658 0
[pid=23661] ppid=23657 vsize=32516 CPUtime=1201.72 cores=2
/proc/23661/stat : 23661 (lingeling) R 23657 23657 22773 0 -1 4202496 10457 0 0 0 120170 2 0 0 25 0 1 0 515360400 33296384 7295 33554432000 4194304 4352236 140736914863824 18446744073709551615 4231116 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/23661/statm: 8129 7295 127 39 0 7209 0
[pid=23662] ppid=23657 vsize=105572 CPUtime=597.15 cores=6
/proc/23662/stat : 23662 (march_hi) R 23657 23657 22773 0 -1 4202496 59768 0 0 0 59706 9 0 0 25 0 1 0 515360400 108105728 22440 33554432000 4194304 4331988 140736142641504 18446744073709551615 4292076 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/23662/statm: 26393 22440 173 34 0 24835 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 3134988

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 23657 and gives
#  childrusage.ru_utime.tv_sec=1201
#  childrusage.ru_utime.tv_usec=707312
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=28995
# CPU time returned by wait4() is 1201.74
# while last known CPU time is 4800.21
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.21
CPU user time (s): 4797.19
CPU system time (s): 3.02
CPU usage (%): 399.447
Max. virtual memory (cumulated for all children) (KiB): 4945448

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1201.71
system time used= 0.028995
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 10980
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 277
involuntary context switches= 2214

runsolver used 1.87172 second user time and 4.68629 second system time

The end

Launcher Data 

Begin job on node136 at 2011-04-29 02:40:24
IDJOB=3190839
IDBENCH=82570
IDSOLVER=1612
FILE ID=node136/3190839-1304037624
RUNJOBID= node136-1304032720-23161
PBS_JOBID= 13170316
Free space on /tmp= 72320 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v40000-c168000-S709305826-098-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3190839-1304037624/watcher-3190839-1304037624 -o /tmp/evaluation-result-3190839-1304037624/solver-3190839-1304037624 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3190839-1304037624.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 83f9a63a559155a858e739be2c598060
RANDOM SEED=1585532463

node136.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      26296784 kB
Buffers:       1048820 kB
Cached:        4905136 kB
SwapCached:        100 kB
Active:         752692 kB
Inactive:      5349592 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      26296784 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:           13224 kB
Writeback:          32 kB
AnonPages:      148440 kB
Mapped:          16792 kB
Slab:           486128 kB
PageTables:       5220 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  1430784 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72320 MiB
End job on node136 at 2011-04-29 03:00:28