Trace number 3264197

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.18 1201.72

General information on the benchmark

NameSAT04/handmade/kexu/frb-0.8/
frb59-26-2.used-as.sat04-892.cnf
MD5SUMfd8d5783ed404e5c6590d627ffdcca50
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1534
Number of clauses132295
Sum of the clauses size266006
Maximum clause length26
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2132236
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 559

Solver Data 

0.00/0.05	This is pico-portfolio, a naive parallel portfolio
0.00/0.05	Copyright (C) 2011 Olivier Roussel
0.00/0.05	Using 4 processing units
0.00/0.05	This portfolio uses the following solvers:
0.00/0.05	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.05	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.05	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.05	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.05	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.05	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.05	Starting clasp 1.3.6 on core(s) 5
0.00/0.05	Starting TNM 2009 on core(s) 7
0.00/0.05	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.05	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3264197-1304093236/watcher-3264197-1304093236 -o /tmp/evaluation-result-3264197-1304093236/solver-3264197-1304093236 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3264197-1304093236.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.32 10.22 10.03 6/182 24851
/proc/meminfo: memFree=12343084/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0

[startup+0.220788 s]
/proc/loadavg: 10.32 10.22 10.03 6/182 24851
/proc/meminfo: memFree=12343084/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300776 s]
/proc/loadavg: 10.32 10.22 10.03 6/182 24851
/proc/meminfo: memFree=12343084/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 23 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.400758 s]
/proc/loadavg: 10.32 10.22 10.03 6/182 24851
/proc/meminfo: memFree=12343084/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 23 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700706 s]
/proc/loadavg: 10.32 10.22 10.03 6/182 24851
/proc/meminfo: memFree=12343084/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 19 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50056 s]
/proc/loadavg: 10.29 10.22 10.03 11/188 24857
/proc/meminfo: memFree=12278700/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=44748 CPUtime=1.49 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 9007 0 0 0 148 1 0 0 25 0 1 0 599554224 45821952 6460 33554432000 4194304 4587354 140734649702992 18446744073709551615 4243904 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 11187 6460 326 96 0 7749 0
[pid=24853] ppid=24851 vsize=20908 CPUtime=1.49 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 3427 0 0 0 149 0 0 0 25 0 1 0 599554224 21409792 2572 33554432000 4194304 4785740 140734040977408 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 5227 2572 340 145 0 2272 0
[pid=24854] ppid=24851 vsize=459124 CPUtime=0.7 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3270 0 0 0 70 0 0 0 25 0 1 0 599554224 470142976 3166 33554432000 134512640 134534564 4286585792 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 114781 3166 95 6 0 113447 0
[pid=24855] ppid=24851 vsize=10900 CPUtime=1.49 cores=3
/proc/24855/stat : 24855 (lingeling) R 24851 24851 24128 0 -1 4202496 2505 0 0 0 149 0 0 0 25 0 1 0 599554224 11161600 1884 33554432000 4194304 4352236 140736422447776 18446744073709551615 4309534 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24855/statm: 2725 1884 124 39 0 1805 0
[pid=24856] ppid=24851 vsize=25784 CPUtime=0.76 cores=7
/proc/24856/stat : 24856 (march_hi) R 24851 24851 24128 0 -1 4202496 10155 0 0 0 75 1 0 0 25 0 1 0 599554224 26402816 2700 33554432000 4194304 4331988 140736393861984 18446744073709551615 4246712 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24856/statm: 6446 2700 175 34 0 4888 0
Current children cumulated CPU time (s) 5.93
Current children cumulated vsize (KiB) 579788

[startup+3.10129 s]
/proc/loadavg: 10.29 10.22 10.03 11/188 24857
/proc/meminfo: memFree=12272996/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=1663596 CPUtime=3.09 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 10053 0 0 0 308 1 0 0 25 0 1 0 599554224 1703522304 6422 33554432000 4194304 4587354 140734649702992 18446744073709551615 4422514 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 415899 6422 337 96 0 412461 0
[pid=24853] ppid=24851 vsize=25824 CPUtime=3.09 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 4648 0 0 0 308 1 0 0 25 0 1 0 599554224 26443776 3793 33554432000 4194304 4785740 140734040977408 18446744073709551615 4552653 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 6456 3793 340 145 0 3501 0
[pid=24854] ppid=24851 vsize=462956 CPUtime=1.5 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3271 0 0 0 150 0 0 0 25 0 1 0 599554224 474066944 3167 33554432000 134512640 134534564 4286585792 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 115739 3167 95 6 0 114405 0
[pid=24855] ppid=24851 vsize=13972 CPUtime=3.08 cores=3
/proc/24855/stat : 24855 (lingeling) R 24851 24851 24128 0 -1 4202496 3276 0 0 0 308 0 0 0 25 0 1 0 599554224 14307328 2655 33554432000 4194304 4352236 140736422447776 18446744073709551615 4231272 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24855/statm: 3493 2655 127 39 0 2573 0
[pid=24856] ppid=24851 vsize=26060 CPUtime=1.56 cores=7
/proc/24856/stat : 24856 (march_hi) R 24851 24851 24128 0 -1 4202496 10217 0 0 0 155 1 0 0 25 0 1 0 599554224 26685440 2762 33554432000 4194304 4331988 140736393861984 18446744073709551615 4239560 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24856/statm: 6515 2762 175 34 0 4957 0
Current children cumulated CPU time (s) 12.32
Current children cumulated vsize (KiB) 2210732

[startup+6.30073 s]
/proc/loadavg: 10.29 10.22 10.03 11/188 24857
/proc/meminfo: memFree=12241772/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=1683380 CPUtime=6.29 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 14949 0 0 0 627 2 0 0 25 0 1 0 599554224 1723781120 11309 33554432000 4194304 4587354 140734649702992 18446744073709551615 4245190 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 420845 11309 337 96 0 417407 0
[pid=24853] ppid=24851 vsize=34164 CPUtime=6.28 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 6711 0 0 0 627 1 0 0 25 0 1 0 599554224 34983936 5856 33554432000 4194304 4785740 140734040977408 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 8541 5856 340 145 0 5586 0
[pid=24854] ppid=24851 vsize=462956 CPUtime=3.1 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3271 0 0 0 310 0 0 0 25 0 1 0 599554224 474066944 3167 33554432000 134512640 134534564 4286585792 18446744073709551615 134528710 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 115739 3167 95 6 0 114405 0
[pid=24855] ppid=24851 vsize=16916 CPUtime=6.28 cores=3
/proc/24855/stat : 24855 (lingeling) R 24851 24851 24128 0 -1 4202496 3841 0 0 0 628 0 0 0 25 0 1 0 599554224 17321984 3220 33554432000 4194304 4352236 140736422447776 18446744073709551615 4328000 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24855/statm: 4229 3220 127 39 0 3309 0
[pid=24856] ppid=24851 vsize=26412 CPUtime=3.14 cores=7
/proc/24856/stat : 24856 (march_hi) R 24851 24851 24128 0 -1 4202496 10307 0 0 0 313 1 0 0 25 0 1 0 599554224 27045888 2852 33554432000 4194304 4331988 140736393861984 18446744073709551615 4237350 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24856/statm: 6603 2852 175 34 0 5045 0
Current children cumulated CPU time (s) 25.09
Current children cumulated vsize (KiB) 2242152

[startup+12.7006 s]
/proc/loadavg: 10.25 10.21 10.03 11/188 24857
/proc/meminfo: memFree=12211040/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=1695828 CPUtime=12.68 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 19033 0 0 0 1266 2 0 0 25 0 1 0 599554224 1736527872 11985 33554432000 4194304 4587354 140734649702992 18446744073709551615 4388264 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 423957 11985 337 96 0 420519 0
[pid=24853] ppid=24851 vsize=37732 CPUtime=12.68 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 7594 0 0 0 1267 1 0 0 25 0 1 0 599554224 38637568 6739 33554432000 4194304 4785740 140734040977408 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 9433 6739 340 145 0 6478 0
[pid=24854] ppid=24851 vsize=462956 CPUtime=6.3 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3271 0 0 0 630 0 0 0 25 0 1 0 599554224 474066944 3167 33554432000 134512640 134534564 4286585792 18446744073709551615 134530793 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 115739 3167 95 6 0 114405 0
[pid=24855] ppid=24851 vsize=20040 CPUtime=12.68 cores=3
/proc/24855/stat : 24855 (lingeling) R 24851 24851 24128 0 -1 4202496 4538 0 0 0 1268 0 0 0 25 0 1 0 599554224 20520960 3917 33554432000 4194304 4352236 140736422447776 18446744073709551615 4328039 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24855/statm: 5010 3917 127 39 0 4090 0
[pid=24856] ppid=24851 vsize=27284 CPUtime=6.3 cores=7
/proc/24856/stat : 24856 (march_hi) R 24851 24851 24128 0 -1 4202496 10513 0 0 0 629 1 0 0 25 0 1 0 599554224 27938816 3058 33554432000 4194304 4331988 140736393861984 18446744073709551615 4237393 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24856/statm: 6821 3058 175 34 0 5263 0
Current children cumulated CPU time (s) 50.64
Current children cumulated vsize (KiB) 2262164

[startup+25.5074 s]
/proc/loadavg: 10.21 10.20 10.03 11/188 24857
/proc/meminfo: memFree=12134448/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=1710332 CPUtime=25.5 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 28493 0 0 0 2546 4 0 0 25 0 1 0 599554224 1751379968 19525 33554432000 4194304 4587354 140734649702992 18446744073709551615 4250248 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 427583 19525 337 96 0 424145 0
[pid=24853] ppid=24851 vsize=43136 CPUtime=25.49 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 8942 0 0 0 2547 2 0 0 25 0 1 0 599554224 44171264 8087 33554432000 4194304 4785740 140734040977408 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 10784 8087 340 145 0 7829 0
[pid=24854] ppid=24851 vsize=462956 CPUtime=12.65 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3271 0 0 0 1265 0 0 0 25 0 1 0 599554224 474066944 3167 33554432000 134512640 134534564 4286585792 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 115739 3167 95 6 0 114405 0
[pid=24855] ppid=24851 vsize=25956 CPUtime=25.5 cores=3
/proc/24855/stat : 24855 (lingeling) R 24851 24851 24128 0 -1 4202496 6017 0 0 0 2549 1 0 0 25 0 1 0 599554224 26578944 5396 33554432000 4194304 4352236 140736422447776 18446744073709551615 4328544 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24855/statm: 6489 5396 127 39 0 5569 0
[pid=24856] ppid=24851 vsize=29236 CPUtime=12.69 cores=7
/proc/24856/stat : 24856 (march_hi) R 24851 24851 24128 0 -1 4202496 10907 0 0 0 1267 2 0 0 25 0 1 0 599554224 29937664 3452 33554432000 4194304 4331988 140736393861984 18446744073709551615 4263238 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24856/statm: 7309 3452 175 34 0 5751 0
Current children cumulated CPU time (s) 101.83
Current children cumulated vsize (KiB) 2289940

[startup+51.1009 s]
/proc/loadavg: 10.20 10.20 10.03 11/188 24858
/proc/meminfo: memFree=12079232/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=1726428 CPUtime=51.08 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 31521 0 0 0 5104 4 0 0 25 0 1 0 599554224 1767862272 22548 33554432000 4194304 4587354 140734649702992 18446744073709551615 4236866 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 431607 22548 337 96 0 428169 0
[pid=24853] ppid=24851 vsize=51084 CPUtime=51.09 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 10844 0 0 0 5106 3 0 0 25 0 1 0 599554224 52310016 9989 33554432000 4194304 4785740 140734040977408 18446744073709551615 4548880 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 12771 9989 340 145 0 9816 0
[pid=24854] ppid=24851 vsize=462956 CPUtime=25.4 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3271 0 0 0 2540 0 0 0 25 0 1 0 599554224 474066944 3167 33554432000 134512640 134534564 4286585792 18446744073709551615 134528737 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 115739 3167 95 6 0 114405 0
[pid=24855] ppid=24851 vsize=29028 CPUtime=51.09 cores=3

################
# More data... #
################

/proc/24856/statm: 25948 18958 175 34 0 24390 0
Current children cumulated CPU time (s) 3763.98
Current children cumulated vsize (KiB) 2553600

[startup+1002.31 s]
/proc/loadavg: 10.24 10.20 10.11 11/188 24916
/proc/meminfo: memFree=12233184/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=1774300 CPUtime=1002.21 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 128338 0 0 0 100169 52 0 0 25 0 1 0 599554224 1816883200 36067 33554432000 4194304 4587354 140734649702992 18446744073709551615 4238593 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 443575 36067 337 96 0 440137 0
[pid=24853] ppid=24851 vsize=133992 CPUtime=1002.32 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 30129 0 0 0 100223 9 0 0 25 0 1 0 599554224 137207808 29274 33554432000 4194304 4785740 140734040977408 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 33498 29274 340 145 0 30543 0
[pid=24854] ppid=24851 vsize=462956 CPUtime=498.42 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3271 0 0 0 49842 0 0 0 25 0 1 0 599554224 474066944 3167 33554432000 134512640 134534564 4286585792 18446744073709551615 134529632 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 115739 3167 95 6 0 114405 0
[pid=24855] ppid=24851 vsize=76156 CPUtime=1002.32 cores=3
/proc/24855/stat : 24855 (lingeling) R 24851 24851 24128 0 -1 4202496 167686 0 0 0 100209 23 0 0 25 0 1 0 599554224 77983744 18201 33554432000 4194304 4352236 140736422447776 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24855/statm: 19039 18201 127 39 0 18119 0
[pid=24856] ppid=24851 vsize=116888 CPUtime=498.41 cores=7
/proc/24856/stat : 24856 (march_hi) R 24851 24851 24128 0 -1 4202496 28042 0 0 0 49829 12 0 0 25 0 1 0 599554224 119693312 20036 33554432000 4194304 4331988 140736393861984 18446744073709551615 4228224 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24856/statm: 29222 20036 175 34 0 27664 0
Current children cumulated CPU time (s) 4003.68
Current children cumulated vsize (KiB) 2582616

[startup+1062.3 s]
/proc/loadavg: 10.33 10.23 10.12 11/188 24918
/proc/meminfo: memFree=12226852/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=1775344 CPUtime=1062.21 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 135310 0 0 0 106163 58 0 0 25 0 1 0 599554224 1817952256 36067 33554432000 4194304 4587354 140734649702992 18446744073709551615 4238512 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 443836 36067 337 96 0 440398 0
[pid=24853] ppid=24851 vsize=137232 CPUtime=1062.32 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 30864 0 0 0 106223 9 0 0 25 0 1 0 599554224 140525568 30009 33554432000 4194304 4785740 140734040977408 18446744073709551615 4287038 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 34308 30009 340 145 0 31353 0
[pid=24854] ppid=24851 vsize=462956 CPUtime=528.22 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3271 0 0 0 52822 0 0 0 25 0 1 0 599554224 474066944 3167 33554432000 134512640 134534564 4286585792 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 115739 3167 95 6 0 114405 0
[pid=24855] ppid=24851 vsize=61820 CPUtime=1062.31 cores=3
/proc/24855/stat : 24855 (lingeling) R 24851 24851 24128 0 -1 4202496 175878 0 0 0 106207 24 0 0 25 0 1 0 599554224 63303680 14617 33554432000 4194304 4352236 140736422447776 18446744073709551615 4328366 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24855/statm: 15455 14617 127 39 0 14535 0
[pid=24856] ppid=24851 vsize=116888 CPUtime=528.29 cores=7
/proc/24856/stat : 24856 (march_hi) R 24851 24851 24128 0 -1 4202496 28977 0 0 0 52816 13 0 0 25 0 1 0 599554224 119693312 20971 33554432000 4194304 4331988 140736393861984 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24856/statm: 29222 20971 175 34 0 27664 0
Current children cumulated CPU time (s) 4243.35
Current children cumulated vsize (KiB) 2572564

[startup+1122.3 s]
/proc/loadavg: 10.28 10.23 10.12 11/188 24920
/proc/meminfo: memFree=12211344/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=1768808 CPUtime=1122.21 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 142654 0 0 0 112159 62 0 0 25 0 1 0 599554224 1811259392 36067 33554432000 4194304 4587354 140734649702992 18446744073709551615 4239793 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 442202 36067 337 96 0 438764 0
[pid=24853] ppid=24851 vsize=140112 CPUtime=1122.32 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 31542 0 0 0 112223 9 0 0 25 0 1 0 599554224 143474688 30687 33554432000 4194304 4785740 140734040977408 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 35028 30687 340 145 0 32073 0
[pid=24854] ppid=24851 vsize=462956 CPUtime=558.07 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3271 0 0 0 55807 0 0 0 25 0 1 0 599554224 474066944 3167 33554432000 134512640 134534564 4286585792 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 115739 3167 95 6 0 114405 0
[pid=24855] ppid=24851 vsize=66940 CPUtime=1122.31 cores=3
/proc/24855/stat : 24855 (lingeling) R 24851 24851 24128 0 -1 4202496 187398 0 0 0 112206 25 0 0 25 0 1 0 599554224 68546560 15897 33554432000 4194304 4352236 140736422447776 18446744073709551615 4329631 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24855/statm: 16735 15897 127 39 0 15815 0
[pid=24856] ppid=24851 vsize=116888 CPUtime=558.11 cores=7
/proc/24856/stat : 24856 (march_hi) R 24851 24851 24128 0 -1 4202496 30043 0 0 0 55797 14 0 0 25 0 1 0 599554224 119693312 22037 33554432000 4194304 4331988 140736393861984 18446744073709551615 4228235 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24856/statm: 29222 22037 175 34 0 27664 0
Current children cumulated CPU time (s) 4483.02
Current children cumulated vsize (KiB) 2574028

[startup+1182.3 s]
/proc/loadavg: 10.22 10.21 10.12 11/188 24921
/proc/meminfo: memFree=12194220/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=1780232 CPUtime=1182.19 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 150174 0 0 0 118152 67 0 0 25 0 1 0 599554224 1822957568 36067 33554432000 4194304 4587354 140734649702992 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 445058 36067 337 96 0 441620 0
[pid=24853] ppid=24851 vsize=141536 CPUtime=1182.32 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 31873 0 0 0 118223 9 0 0 25 0 1 0 599554224 144932864 31018 33554432000 4194304 4785740 140734040977408 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 35384 31018 340 145 0 32429 0
[pid=24854] ppid=24851 vsize=462956 CPUtime=587.91 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3271 0 0 0 58791 0 0 0 25 0 1 0 599554224 474066944 3167 33554432000 134512640 134534564 4286585792 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 115739 3167 95 6 0 114405 0
[pid=24855] ppid=24851 vsize=66940 CPUtime=1182.31 cores=3
/proc/24855/stat : 24855 (lingeling) R 24851 24851 24128 0 -1 4202496 196358 0 0 0 118205 26 0 0 25 0 1 0 599554224 68546560 15897 33554432000 4194304 4352236 140736422447776 18446744073709551615 4328308 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24855/statm: 16735 15897 127 39 0 15815 0
[pid=24856] ppid=24851 vsize=116888 CPUtime=587.95 cores=7
/proc/24856/stat : 24856 (march_hi) R 24851 24851 24128 0 -1 4202496 31201 0 0 0 58780 15 0 0 25 0 1 0 599554224 119693312 23195 33554432000 4194304 4331988 140736393861984 18446744073709551615 4228270 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24856/statm: 29222 23195 175 34 0 27664 0
Current children cumulated CPU time (s) 4722.68
Current children cumulated vsize (KiB) 2586876



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 10.23 10.22 10.12 11/188 24922
/proc/meminfo: memFree=12193972/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=1780232 CPUtime=1201.6 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 152618 0 0 0 120091 69 0 0 25 0 1 0 599554224 1822957568 38440 33554432000 4194304 4587354 140734649702992 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 445058 38440 337 96 0 441620 0
[pid=24853] ppid=24851 vsize=141536 CPUtime=1201.72 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 31907 0 0 0 120163 9 0 0 25 0 1 0 599554224 144932864 31052 33554432000 4194304 4785740 140734040977408 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 35384 31052 340 145 0 32429 0
[pid=24854] ppid=24851 vsize=462956 CPUtime=597.55 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3271 0 0 0 59755 0 0 0 25 0 1 0 599554224 474066944 3167 33554432000 134512640 134534564 4286585792 18446744073709551615 134530573 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 115739 3167 95 6 0 114405 0
[pid=24855] ppid=24851 vsize=60796 CPUtime=1201.71 cores=3
/proc/24855/stat : 24855 (lingeling) R 24851 24851 24128 0 -1 4202496 199686 0 0 0 120145 26 0 0 25 0 1 0 599554224 62255104 14361 33554432000 4194304 4352236 140736422447776 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24855/statm: 15199 14361 127 39 0 14279 0
[pid=24856] ppid=24851 vsize=116888 CPUtime=597.6 cores=7
/proc/24856/stat : 24856 (march_hi) R 24851 24851 24128 0 -1 4202496 31497 0 0 0 59745 15 0 0 25 0 1 0 599554224 119693312 23491 33554432000 4194304 4331988 140736393861984 18446744073709551615 4315832 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24856/statm: 29222 23491 175 34 0 27664 0
Current children cumulated CPU time (s) 4800.18
Current children cumulated vsize (KiB) 2580732

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 10.23 10.22 10.12 11/188 24922
/proc/meminfo: memFree=12193972/32951124 swapFree=67102140/67111528
[pid=24851] ppid=24849 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24851/stat : 24851 (ppfolio) S 24849 24851 24128 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599554224 18763776 390 33554432000 4194304 4287625 140734781437504 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 3 0 0 0
/proc/24851/statm: 4581 390 341 23 0 91 0
[pid=24852] ppid=24851 vsize=1780232 CPUtime=1201.6 cores=1
/proc/24852/stat : 24852 (cryptominisat) R 24851 24851 24128 0 -1 4202496 152618 0 0 0 120091 69 0 0 25 0 1 0 599554224 1822957568 38440 33554432000 4194304 4587354 140734649702992 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24852/statm: 445058 38440 337 96 0 441620 0
[pid=24853] ppid=24851 vsize=141536 CPUtime=1201.72 cores=5
/proc/24853/stat : 24853 (clasp) R 24851 24851 24128 0 -1 4202496 31907 0 0 0 120163 9 0 0 25 0 1 0 599554224 144932864 31052 33554432000 4194304 4785740 140734040977408 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24853/statm: 35384 31052 340 145 0 32429 0
[pid=24854] ppid=24851 vsize=462956 CPUtime=597.55 cores=7
/proc/24854/stat : 24854 (TNM) R 24851 24851 24128 0 -1 4202496 3271 0 0 0 59755 0 0 0 25 0 1 0 599554224 474066944 3167 33554432000 134512640 134534564 4286585792 18446744073709551615 134530573 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24854/statm: 115739 3167 95 6 0 114405 0
[pid=24855] ppid=24851 vsize=60796 CPUtime=1201.71 cores=3
/proc/24855/stat : 24855 (lingeling) R 24851 24851 24128 0 -1 4202496 199686 0 0 0 120145 26 0 0 25 0 1 0 599554224 62255104 14361 33554432000 4194304 4352236 140736422447776 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24855/statm: 15199 14361 127 39 0 14279 0
[pid=24856] ppid=24851 vsize=116888 CPUtime=597.6 cores=7
/proc/24856/stat : 24856 (march_hi) R 24851 24851 24128 0 -1 4202496 31497 0 0 0 59745 15 0 0 25 0 1 0 599554224 119693312 23491 33554432000 4194304 4331988 140736393861984 18446744073709551615 4315832 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24856/statm: 29222 23491 175 34 0 27664 0
Current children cumulated CPU time (s) 4800.18
Current children cumulated vsize (KiB) 2580732

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 24851 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=2999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=3999
# CPU time returned by wait4() is 0.006998
# while last known CPU time is 4800.18
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.72
CPU time (s): 4800.18
CPU user time (s): 4798.99
CPU system time (s): 1.19
CPU usage (%): 399.444
Max. virtual memory (cumulated for all children) (KiB): 4192060

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.002999
system time used= 0.003999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 523
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 378
involuntary context switches= 6

runsolver used 1.79773 second user time and 4.76628 second system time

The end

Launcher Data 

Begin job on node119 at 2011-04-29 18:07:16
IDJOB=3264197
IDBENCH=84560
IDSOLVER=1612
FILE ID=node119/3264197-1304093236
RUNJOBID= node119-1304090423-24478
PBS_JOBID= 13170550
Free space on /tmp= 71804 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT04/handmade/kexu/frb-0.8/frb59-26-2.used-as.sat04-892.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3264197-1304093236/watcher-3264197-1304093236 -o /tmp/evaluation-result-3264197-1304093236/solver-3264197-1304093236 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3264197-1304093236.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= fd8d5783ed404e5c6590d627ffdcca50
RANDOM SEED=496831602

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      12343240 kB
Buffers:       2235204 kB
Cached:       17327232 kB
SwapCached:       5504 kB
Active:        8390828 kB
Inactive:     11630248 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      12343240 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:           11660 kB
Writeback:           0 kB
AnonPages:      455520 kB
Mapped:          16800 kB
Slab:           521840 kB
PageTables:       5944 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  1266496 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71796 MiB
End job on node119 at 2011-04-29 18:27:19