Trace number 3191567

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio seq? (TO) 1300 1300.01

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v50000-c210000-S1170024351-015-UNKNOWN.cnf
MD5SUM0f7796213ec78265bd44aa83c0fa283d
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark153.855
Satisfiable
(Un)Satisfiability was proved
Number of variables50000
Number of clauses210000
Sum of the clauses size630000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3210000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.01	This is pico-portfolio, a naive parallel portfolio
0.00/0.01	Copyright (C) 2011 Olivier Roussel
0.00/0.01	Using 1 processing units
0.00/0.01	This portfolio uses the following solvers:
0.00/0.01	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.01	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.01	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.01	Starting cryptominisat 2.7.1 on core(s) 5
0.00/0.01	Starting clasp 1.3.6 on core(s) 5
0.00/0.01	Starting TNM 2009 on core(s) 5

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 893)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3191567-1302538351/watcher-3191567-1302538351 -o /tmp/evaluation-result-3191567-1302538351/solver-3191567-1302538351 -C 2400 -W 1300 -M 7680 ppfolio --nbcore=1 HOME/instance-3191567-1302538351.cnf 

running on 2 cores: 5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 11.59 11.71 9.21 11/198 4087
/proc/meminfo: memFree=13751000/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=11760 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (runsolver) R 4085 4087 3790 0 -1 4202560 40 0 0 0 0 0 0 0 25 0 1 0 365434969 12042240 49 33554432000 4194304 5392636 140736729543744 18446744073709551615 4838247 0 2147483391 4096 24578 0 0 0 17 7 0 0 0
/proc/4087/statm: 2940 49 30 293 0 2644 0

[startup+0.054219 s]
/proc/loadavg: 11.59 11.71 9.21 11/198 4087
/proc/meminfo: memFree=13751000/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 500 0 0 0 0 0 0 0 25 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 4519424627646103816 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100211 s]
/proc/loadavg: 11.59 11.71 9.21 11/198 4087
/proc/meminfo: memFree=13751000/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 500 0 0 0 0 0 0 0 25 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 4519424627646103816 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300177 s]
/proc/loadavg: 11.59 11.71 9.21 11/198 4087
/proc/meminfo: memFree=13751000/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 24 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631368841419292672 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.701105 s]
/proc/loadavg: 11.59 11.71 9.21 11/198 4087
/proc/meminfo: memFree=13751000/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 21 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 0 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50097 s]
/proc/loadavg: 11.59 11.71 9.21 13/202 4091
/proc/meminfo: memFree=13672200/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631369803491966976 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=62288 CPUtime=0.5 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 11474 0 0 0 49 1 0 0 25 0 1 0 365434970 63782912 10756 33554432000 4194304 4587354 140735413130144 18446744073709551615 4205152 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 15572 10756 312 96 0 12134 0
[pid=4090] ppid=4087 vsize=41936 CPUtime=0.39 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 9205 0 0 0 38 1 0 0 25 0 1 0 365434970 42942464 7383 33554432000 4194304 4785740 140734917936128 18446744073709551615 4499974 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 10484 7383 319 145 0 7529 0
[pid=4091] ppid=4087 vsize=466780 CPUtime=0.58 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 5019 0 0 0 57 1 0 0 25 0 1 0 365434970 477982720 4916 33554432000 134512640 134534564 4294004240 18446744073709551615 134523484 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 116695 4916 91 6 0 115361 0
Current children cumulated CPU time (s) 1.47
Current children cumulated vsize (KiB) 589328

[startup+3.10074 s]
/proc/loadavg: 11.59 11.71 9.21 13/202 4091
/proc/meminfo: memFree=13621360/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 0 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=71872 CPUtime=1 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 16447 0 0 0 98 2 0 0 25 0 1 0 365434970 73596928 12479 33554432000 4194304 4587354 140735413130144 18446744073709551615 4237493 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 17968 12479 325 96 0 14530 0
[pid=4090] ppid=4087 vsize=39264 CPUtime=0.97 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 9701 0 0 0 96 1 0 0 25 0 1 0 365434970 40206336 6979 33554432000 4194304 4785740 140734917936128 18446744073709551615 4553075 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 9816 6979 340 145 0 6861 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=1.09 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 108 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134528504 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 3.06
Current children cumulated vsize (KiB) 601520

[startup+6.30112 s]
/proc/loadavg: 11.63 11.71 9.22 13/202 4092
/proc/meminfo: memFree=13602964/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 0 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=71872 CPUtime=2.09 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 17176 0 0 0 207 2 0 0 25 0 1 0 365434970 73596928 13206 33554432000 4194304 4587354 140735413130144 18446744073709551615 4238323 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 17968 13206 326 96 0 14530 0
[pid=4090] ppid=4087 vsize=39264 CPUtime=1.99 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 9701 0 0 0 198 1 0 0 25 0 1 0 365434970 40206336 6979 33554432000 4194304 4785740 140734917936128 18446744073709551615 248089394955 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 9816 6979 340 145 0 6861 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=2.17 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 216 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134525904 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 6.25
Current children cumulated vsize (KiB) 601520

[startup+12.701 s]
/proc/loadavg: 11.66 11.72 9.24 13/202 4092
/proc/meminfo: memFree=13525416/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631368978858246144 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=109376 CPUtime=4.19 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 18378 0 0 0 417 2 0 0 25 0 1 0 365434970 112001024 14402 33554432000 4194304 4587354 140735413130144 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 27344 14402 326 96 0 23906 0
[pid=4090] ppid=4087 vsize=43576 CPUtime=4.18 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 10859 0 0 0 416 2 0 0 25 0 1 0 365434970 44621824 8137 33554432000 4194304 4785740 140734917936128 18446744073709551615 4549837 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 10894 8137 340 145 0 7939 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=4.29 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 428 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 12.66
Current children cumulated vsize (KiB) 643336

[startup+25.5007 s]
/proc/loadavg: 11.73 11.73 9.28 13/202 4092
/proc/meminfo: memFree=13473556/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631369253736153088 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=109376 CPUtime=8.49 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 20630 0 0 0 846 3 0 0 25 0 1 0 365434970 112001024 16649 33554432000 4194304 4587354 140735413130144 18446744073709551615 4238728 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 27344 16649 326 96 0 23906 0
[pid=4090] ppid=4087 vsize=58636 CPUtime=8.39 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 14635 0 0 0 836 3 0 0 25 0 1 0 365434970 60043264 11913 33554432000 4194304 4785740 140734917936128 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 14659 11913 340 145 0 11704 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=8.58 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 857 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 25.46
Current children cumulated vsize (KiB) 658396

[startup+51.1012 s]
/proc/loadavg: 11.62 11.70 9.34 13/202 4124
/proc/meminfo: memFree=13957316/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631369803491966976 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=163272 CPUtime=16.99 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 28262 0 0 0 1694 5 0 0 25 0 1 0 365434970 167190528 16774 33554432000 4194304 4587354 140735413130144 18446744073709551615 4241101 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 40818 16774 331 96 0 37380 0
[pid=4090] ppid=4087 vsize=96860 CPUtime=16.98 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 24128 0 0 0 1694 4 0 0 25 0 1 0 365434970 99184640 21406 33554432000 4194304 4785740 140734917936128 18446744073709551615 4546087 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 24215 21406 340 145 0 21260 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=17.09 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 1708 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 51.06
Current children cumulated vsize (KiB) 750516

[startup+102.307 s]
/proc/loadavg: 11.83 11.75 9.47 13/202 4125
/proc/meminfo: memFree=13763236/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631369940930920448 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=992112 CPUtime=34.09 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 81254 0 0 0 3395 14 0 0 25 0 1 0 365434970 1015922688 20409 33554432000 4194304 4587354 140735413130144 18446744073709551615 4239464 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 248028 20409 337 96 0 244590 0
[pid=4090] ppid=4087 vsize=161404 CPUtime=34 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 40148 0 0 0 3393 7 0 0 25 0 1 0 365434970 165277696 37426 33554432000 4194304 4785740 140734917936128 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 40351 37426 340 145 0 37396 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=34.19 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 3418 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134528730 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 102.28
Current children cumulated vsize (KiB) 1643900

[startup+162.301 s]
/proc/loadavg: 11.97 11.80 9.63 13/202 4127
/proc/meminfo: memFree=13631036/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631370490686734336 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=169940 CPUtime=54.09 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 105739 0 0 0 5389 20 0 0 25 0 1 0 365434970 174018560 18059 33554432000 4194304 4587354 140735413130144 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 42485 18059 337 96 0 39047 0
[pid=4090] ppid=4087 vsize=203140 CPUtime=54 cores=5

################
# More data... #
################

/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 822.29
Current children cumulated vsize (KiB) 1839764

[startup+882.3 s]
/proc/loadavg: 12.06 12.00 10.90 13/202 4206
/proc/meminfo: memFree=13276836/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631370903003594752 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=992932 CPUtime=294.09 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 340195 0 0 0 29343 66 0 0 25 0 1 0 365434970 1016762368 27944 33554432000 4194304 4587354 140735413130144 18446744073709551615 248089394840 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 248233 27944 337 96 0 244795 0
[pid=4090] ppid=4087 vsize=359936 CPUtime=294.05 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 88760 0 0 0 29387 18 0 0 25 0 1 0 365434970 368574464 86038 33554432000 4194304 4785740 140734917936128 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 89984 86038 340 145 0 87029 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=294.15 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 29414 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134529640 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 882.29
Current children cumulated vsize (KiB) 1843252

[startup+942.301 s]
/proc/loadavg: 12.02 12.00 10.97 13/202 4207
/proc/meminfo: memFree=13246440/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631368978858246144 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=183576 CPUtime=314.1 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 347398 0 0 0 31342 68 0 0 25 0 1 0 365434970 187981824 28098 33554432000 4194304 4587354 140735413130144 18446744073709551615 4238268 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 45894 28098 337 96 0 42456 0
[pid=4090] ppid=4087 vsize=372332 CPUtime=314.05 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 91816 0 0 0 31386 19 0 0 25 0 1 0 365434970 381267968 89094 33554432000 4194304 4785740 140734917936128 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 93083 89094 340 145 0 90128 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=314.14 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 31413 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 942.29
Current children cumulated vsize (KiB) 1046292

[startup+1002.3 s]
/proc/loadavg: 12.01 12.00 11.03 13/202 4240
/proc/meminfo: memFree=13523784/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631369116297199616 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=1035372 CPUtime=334.1 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 359509 0 0 0 33339 71 0 0 25 0 1 0 365434970 1060220928 32825 33554432000 4194304 4587354 140735413130144 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 258843 32825 337 96 0 255405 0
[pid=4090] ppid=4087 vsize=372332 CPUtime=334.05 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 91816 0 0 0 33386 19 0 0 25 0 1 0 365434970 381267968 89094 33554432000 4194304 4785740 140734917936128 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 93083 89094 340 145 0 90128 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=334.14 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 33413 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134530151 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 1002.29
Current children cumulated vsize (KiB) 1898088

[startup+1062.3 s]
/proc/loadavg: 11.56 11.89 11.05 12/201 4272
/proc/meminfo: memFree=13547496/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631369253736153088 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=1055692 CPUtime=354.11 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 366700 0 0 0 35337 74 0 0 25 0 1 0 365434970 1081028608 32347 33554432000 4194304 4587354 140735413130144 18446744073709551615 4238332 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 263923 32347 337 96 0 260485 0
[pid=4090] ppid=4087 vsize=372332 CPUtime=354.04 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 91824 0 0 0 35385 19 0 0 25 0 1 0 365434970 381267968 89102 33554432000 4194304 4785740 140734917936128 18446744073709551615 4287204 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 93083 89102 340 145 0 90128 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=354.14 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 35413 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134528302 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 1062.29
Current children cumulated vsize (KiB) 1918408

[startup+1122.3 s]
/proc/loadavg: 11.20 11.72 11.05 12/201 4273
/proc/meminfo: memFree=13540340/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631369391175106560 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=203332 CPUtime=374.12 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 373828 0 0 0 37336 76 0 0 25 0 1 0 365434970 208211968 31409 33554432000 4194304 4587354 140735413130144 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 50833 31409 337 96 0 47395 0
[pid=4090] ppid=4087 vsize=380548 CPUtime=374.03 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 93884 0 0 0 37384 19 0 0 25 0 1 0 365434970 389681152 91162 33554432000 4194304 4785740 140734917936128 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 95137 91162 340 145 0 92182 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=374.13 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 37412 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134530593 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 1122.28
Current children cumulated vsize (KiB) 1074264

[startup+1182.3 s]
/proc/loadavg: 11.07 11.59 11.04 12/201 4275
/proc/meminfo: memFree=13345880/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 0 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=366688 CPUtime=394.12 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 415190 0 0 0 39331 81 0 0 25 0 1 0 365434970 375488512 72766 33554432000 4194304 4587354 140735413130144 18446744073709551615 4365216 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 91672 72766 337 96 0 88234 0
[pid=4090] ppid=4087 vsize=403788 CPUtime=394.04 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 99589 0 0 0 39382 22 0 0 25 0 1 0 365434970 413478912 96867 33554432000 4194304 4785740 140734917936128 18446744073709551615 4543541 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 100947 96867 340 145 0 97992 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=394.13 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 39412 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134528330 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 1182.29
Current children cumulated vsize (KiB) 1260860

[startup+1242.3 s]
/proc/loadavg: 11.02 11.48 11.03 12/201 4276
/proc/meminfo: memFree=13479532/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631369253736153088 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=1007604 CPUtime=414.13 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 426463 0 0 0 41328 85 0 0 25 0 1 0 365434970 1031786496 30735 33554432000 4194304 4587354 140735413130144 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 251901 30735 337 96 0 248463 0
[pid=4090] ppid=4087 vsize=403788 CPUtime=414.03 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 99589 0 0 0 41381 22 0 0 25 0 1 0 365434970 413478912 96867 33554432000 4194304 4785740 140734917936128 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 100947 96867 340 145 0 97992 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=414.13 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 41412 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134528877 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 1242.29
Current children cumulated vsize (KiB) 1901776



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 11.04 11.40 11.03 12/201 4278
/proc/meminfo: memFree=13472680/32950928 swapFree=67111528/67111528
[pid=4087] ppid=4085 vsize=18324 CPUtime=0 cores=5,7
/proc/4087/stat : 4087 (ppfolio) S 4085 4087 3790 0 -1 4202496 501 0 0 0 0 0 0 0 15 0 1 0 365434969 18763776 394 33554432000 4194304 4287625 140736484038512 18446744073709551615 248089718287 0 0 4096 65536 5631369391175106560 0 0 17 5 0 0 0
/proc/4087/statm: 4581 394 345 23 0 91 0
[pid=4089] ppid=4087 vsize=1015820 CPUtime=433.32 cores=5
/proc/4089/stat : 4089 (cryptominisat) R 4087 4087 3790 0 -1 4202496 433012 0 0 0 43245 87 0 0 25 0 1 0 365434970 1040199680 28778 33554432000 4194304 4587354 140735413130144 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/4089/statm: 253955 28778 337 96 0 250517 0
[pid=4090] ppid=4087 vsize=403788 CPUtime=433.25 cores=5
/proc/4090/stat : 4090 (clasp) R 4087 4087 3790 0 -1 4202496 99589 0 0 0 43303 22 0 0 25 0 1 0 365434970 413478912 96867 33554432000 4194304 4785740 140734917936128 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/4090/statm: 100947 96867 340 145 0 97992 0
[pid=4091] ppid=4087 vsize=472060 CPUtime=433.43 cores=5
/proc/4091/stat : 4091 (TNM) R 4087 4087 3790 0 -1 4202496 8336 0 0 0 43342 1 0 0 25 0 1 0 365434970 483389440 8233 33554432000 134512640 134534564 4294004240 18446744073709551615 134528504 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/4091/statm: 118015 8233 95 6 0 116681 0
Current children cumulated CPU time (s) 1300
Current children cumulated vsize (KiB) 1909992

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 4087 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=1999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=999
# CPU time returned by wait4() is 0.002998
# while last known CPU time is 1300
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1300.01
CPU time (s): 1300
CPU user time (s): 1298.9
CPU system time (s): 1.1
CPU usage (%): 99.999
Max. virtual memory (cumulated for all children) (KiB): 2792956

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.001999
system time used= 0.000999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 501
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 190
involuntary context switches= 4

runsolver used 2.09768 second user time and 5.00624 second system time

The end

Launcher Data 

Begin job on node123 at 2011-04-11 18:12:32
IDJOB=3191567
IDBENCH=82698
IDSOLVER=1613
FILE ID=node123/3191567-1302538351
RUNJOBID= node123-1302537048-3811
PBS_JOBID= 12951127
Free space on /tmp= 71476 MiB

SOLVER NAME= ppfolio seq
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v50000-c210000-S1170024351-015-UNKNOWN.cnf
COMMAND LINE= ppfolio --nbcore=1 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3191567-1302538351/watcher-3191567-1302538351 -o /tmp/evaluation-result-3191567-1302538351/solver-3191567-1302538351 -C 2400 -W 1300 -M 7680  ppfolio --nbcore=1 HOME/instance-3191567-1302538351.cnf

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= 0f7796213ec78265bd44aa83c0fa283d
RANDOM SEED=711889014

node123.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      13751280 kB
Buffers:        677872 kB
Cached:       17366532 kB
SwapCached:          0 kB
Active:       10239104 kB
Inactive:      8554988 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      13751280 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           26408 kB
Writeback:           0 kB
AnonPages:      749452 kB
Mapped:          17568 kB
Slab:           339412 kB
PageTables:       7548 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  4068524 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71472 MiB
End job on node123 at 2011-04-11 18:34:14