Trace number 3263537

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (MO) 1473.98 369.013

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_70_15_90_1-dir.shuffled-as.sat03-136.cnf
MD5SUM5bd7c76aed56f767985526812cdb8e0e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark935.889
Satisfiable
(Un)Satisfiability was proved
Number of variables450
Number of clauses31962
Sum of the clauses size64344
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 231947
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 515

Solver Data 

0.00/0.80	This is pico-portfolio, a naive parallel portfolio
0.00/0.80	Copyright (C) 2011 Olivier Roussel
0.00/0.80	Using 4 processing units
0.00/0.80	This portfolio uses the following solvers:
0.00/0.80	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.80	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.80	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.80	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.80	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.80	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.80	Starting clasp 1.3.6 on core(s) 5
0.00/0.80	Starting TNM 2009 on core(s) 7
0.00/0.80	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.80	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3263537-1304091625/watcher-3263537-1304091625 -o /tmp/evaluation-result-3263537-1304091625/solver-3263537-1304091625 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3263537-1304091625.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.42 10.20 9.38 1/175 24572
/proc/meminfo: memFree=12741116/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) R 24570 24572 24128 0 -1 4202496 518 0 0 0 0 0 0 0 22 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 0 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0

[startup+0.096497 s]
/proc/loadavg: 10.42 10.20 9.38 1/175 24572
/proc/meminfo: memFree=12741116/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) R 24570 24572 24128 0 -1 4202496 518 0 0 0 0 0 0 0 22 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 0 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100494 s]
/proc/loadavg: 10.42 10.20 9.38 1/175 24572
/proc/meminfo: memFree=12741116/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) R 24570 24572 24128 0 -1 4202496 518 0 0 0 0 0 0 0 22 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 0 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300462 s]
/proc/loadavg: 10.42 10.20 9.38 1/175 24572
/proc/meminfo: memFree=12741116/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) R 24570 24572 24128 0 -1 4202496 518 0 0 0 0 0 0 0 22 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 0 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700394 s]
/proc/loadavg: 10.42 10.20 9.38 1/175 24572
/proc/meminfo: memFree=12741116/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) R 24570 24572 24128 0 -1 4202496 518 0 0 0 0 0 0 0 22 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 0 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50126 s]
/proc/loadavg: 10.42 10.20 9.38 6/181 24578
/proc/meminfo: memFree=12713304/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=926028 CPUtime=1.49 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 4508 0 0 0 149 0 0 0 20 0 1 0 599393204 948252672 2095 33554432000 4194304 4587354 140734776412336 18446744073709551615 216734875330 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 231507 2095 327 96 0 228069 0
[pid=24574] ppid=24572 vsize=22848 CPUtime=1.48 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 3317 0 0 0 148 0 0 0 25 0 1 0 599393204 23396352 3052 33554432000 4194304 4785740 140736552748448 18446744073709551615 4553440 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 5712 3052 340 145 0 2757 0
[pid=24575] ppid=24572 vsize=453580 CPUtime=0.78 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 996 0 0 0 78 0 0 0 25 0 1 0 599393204 464465920 893 33554432000 134512640 134534564 4291023728 18446744073709551615 134528902 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 113395 893 95 6 0 112061 0
[pid=24576] ppid=24572 vsize=8588 CPUtime=1.49 cores=3
/proc/24576/stat : 24576 (lingeling) R 24572 24572 24128 0 -1 4202496 1540 0 0 0 149 0 0 0 20 0 1 0 599393204 8794112 1308 33554432000 4194304 4352236 140734812281360 18446744073709551615 4328205 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24576/statm: 2147 1308 126 39 0 1227 0
[pid=24577] ppid=24572 vsize=11592 CPUtime=0.7 cores=7
/proc/24577/stat : 24577 (march_hi) R 24572 24572 24128 0 -1 4202496 2355 0 0 0 70 0 0 0 25 0 1 0 599393204 11870208 898 33554432000 4194304 4331988 140734352815504 18446744073709551615 4226471 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24577/statm: 2898 898 174 34 0 1340 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 1440960

[startup+3.10097 s]
/proc/loadavg: 9.99 10.12 9.35 11/187 24609
/proc/meminfo: memFree=12687564/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=922240 CPUtime=3.09 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 6332 0 0 0 308 1 0 0 25 0 1 0 599393204 944373760 2681 33554432000 4194304 4587354 140734776412336 18446744073709551615 4238571 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 230560 2681 327 96 0 227122 0
[pid=24574] ppid=24572 vsize=26140 CPUtime=3.08 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 4123 0 0 0 308 0 0 0 25 0 1 0 599393204 26767360 3858 33554432000 4194304 4785740 140736552748448 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 6535 3858 340 145 0 3580 0
[pid=24575] ppid=24572 vsize=457412 CPUtime=1.57 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 997 0 0 0 157 0 0 0 25 0 1 0 599393204 468389888 894 33554432000 134512640 134534564 4291023728 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 114353 894 95 6 0 113019 0
[pid=24576] ppid=24572 vsize=10428 CPUtime=3.09 cores=3
/proc/24576/stat : 24576 (lingeling) R 24572 24572 24128 0 -1 4202496 2207 0 0 0 309 0 0 0 25 0 1 0 599393204 10678272 1768 33554432000 4194304 4352236 140734812281360 18446744073709551615 4234441 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24576/statm: 2607 1768 126 39 0 1687 0
[pid=24577] ppid=24572 vsize=11728 CPUtime=1.5 cores=7
/proc/24577/stat : 24577 (march_hi) R 24572 24572 24128 0 -1 4202496 2381 0 0 0 150 0 0 0 25 0 1 0 599393204 12009472 924 33554432000 4194304 4331988 140734352815504 18446744073709551615 4226217 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24577/statm: 2932 924 174 34 0 1374 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 1446272

[startup+6.30042 s]
/proc/loadavg: 9.99 10.12 9.35 11/188 24610
/proc/meminfo: memFree=12628936/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=945956 CPUtime=6.29 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 12439 0 0 0 628 1 0 0 25 0 1 0 599393204 968658944 7513 33554432000 4194304 4587354 140734776412336 18446744073709551615 4365352 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 236489 7513 337 96 0 233051 0
[pid=24574] ppid=24572 vsize=31748 CPUtime=6.29 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 5450 0 0 0 628 1 0 0 25 0 1 0 599393204 32509952 5185 33554432000 4194304 4785740 140736552748448 18446744073709551615 4287058 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 7937 5185 340 145 0 4982 0
[pid=24575] ppid=24572 vsize=457412 CPUtime=3.15 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 997 0 0 0 315 0 0 0 25 0 1 0 599393204 468389888 894 33554432000 134512640 134534564 4291023728 18446744073709551615 134528877 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 114353 894 95 6 0 113019 0
[pid=24576] ppid=24572 vsize=13916 CPUtime=6.29 cores=3
/proc/24576/stat : 24576 (lingeling) R 24572 24572 24128 0 -1 4202496 3745 0 0 0 629 0 0 0 25 0 1 0 599393204 14249984 2640 33554432000 4194304 4352236 140734812281360 18446744073709551615 4278692 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24576/statm: 3479 2640 126 39 0 2559 0
[pid=24577] ppid=24572 vsize=11728 CPUtime=3.1 cores=7
/proc/24577/stat : 24577 (march_hi) R 24572 24572 24128 0 -1 4202496 2401 0 0 0 310 0 0 0 25 0 1 0 599393204 12009472 944 33554432000 4194304 4331988 140734352815504 18446744073709551615 4210448 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24577/statm: 2932 944 174 34 0 1374 0
Current children cumulated CPU time (s) 25.12
Current children cumulated vsize (KiB) 1479084

[startup+12.7003 s]
/proc/loadavg: 9.99 10.11 9.36 11/188 24610
/proc/meminfo: memFree=12573920/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=951300 CPUtime=12.69 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 13099 0 0 0 1268 1 0 0 25 0 1 0 599393204 974131200 6892 33554432000 4194304 4587354 140734776412336 18446744073709551615 4238755 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 237825 6892 337 96 0 234387 0
[pid=24574] ppid=24572 vsize=41192 CPUtime=12.69 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 7703 0 0 0 1268 1 0 0 25 0 1 0 599393204 42180608 7438 33554432000 4194304 4785740 140736552748448 18446744073709551615 4287008 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 10298 7438 340 145 0 7343 0
[pid=24575] ppid=24572 vsize=457412 CPUtime=6.32 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 997 0 0 0 632 0 0 0 25 0 1 0 599393204 468389888 894 33554432000 134512640 134534564 4291023728 18446744073709551615 134528979 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 114353 894 95 6 0 113019 0
[pid=24576] ppid=24572 vsize=14416 CPUtime=12.69 cores=3
/proc/24576/stat : 24576 (lingeling) R 24572 24572 24128 0 -1 4202496 4730 0 0 0 1269 0 0 0 25 0 1 0 599393204 14761984 2765 33554432000 4194304 4352236 140734812281360 18446744073709551615 4328345 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24576/statm: 3604 2765 126 39 0 2684 0
[pid=24577] ppid=24572 vsize=12184 CPUtime=6.3 cores=7
/proc/24577/stat : 24577 (march_hi) R 24572 24572 24128 0 -1 4202496 2512 0 0 0 630 0 0 0 25 0 1 0 599393204 12476416 1055 33554432000 4194304 4331988 140734352815504 18446744073709551615 4246730 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24577/statm: 3046 1055 175 34 0 1488 0
Current children cumulated CPU time (s) 50.69
Current children cumulated vsize (KiB) 1494828

[startup+25.5011 s]
/proc/loadavg: 9.99 10.11 9.37 11/188 24610
/proc/meminfo: memFree=12527452/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=7839556 CPUtime=25.49 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 18042 0 0 0 2547 2 0 0 25 0 1 0 599393204 8027705344 11827 33554432000 4194304 4587354 140734776412336 18446744073709551615 4243819 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 1959889 11827 337 96 0 1956451 0
[pid=24574] ppid=24572 vsize=49240 CPUtime=25.49 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 9539 0 0 0 2548 1 0 0 25 0 1 0 599393204 50421760 9274 33554432000 4194304 4785740 140736552748448 18446744073709551615 4552823 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 12310 9274 340 145 0 9355 0
[pid=24575] ppid=24572 vsize=457412 CPUtime=12.82 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 997 0 0 0 1282 0 0 0 25 0 1 0 599393204 468389888 894 33554432000 134512640 134534564 4291023728 18446744073709551615 134528926 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 114353 894 95 6 0 113019 0
[pid=24576] ppid=24572 vsize=18520 CPUtime=25.49 cores=3
/proc/24576/stat : 24576 (lingeling) R 24572 24572 24128 0 -1 4202496 8061 0 0 0 2548 1 0 0 25 0 1 0 599393204 18964480 3791 33554432000 4194304 4352236 140734812281360 18446744073709551615 4280991 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24576/statm: 4630 3791 126 39 0 3710 0
[pid=24577] ppid=24572 vsize=13416 CPUtime=12.53 cores=7
/proc/24577/stat : 24577 (march_hi) R 24572 24572 24128 0 -1 4202496 2732 0 0 0 1253 0 0 0 25 0 1 0 599393204 13737984 1275 33554432000 4194304 4331988 140734352815504 18446744073709551615 4237419 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24577/statm: 3354 1275 175 34 0 1796 0
Current children cumulated CPU time (s) 101.82
Current children cumulated vsize (KiB) 8396468

[startup+51.1006 s]
/proc/loadavg: 10.07 10.12 9.39 11/188 24611
/proc/meminfo: memFree=12464780/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=973336 CPUtime=51.09 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 21945 0 0 0 5107 2 0 0 25 0 1 0 599393204 996696064 15723 33554432000 4194304 4587354 140734776412336 18446744073709551615 4238573 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 243334 15723 337 96 0 239896 0
[pid=24574] ppid=24572 vsize=57716 CPUtime=51.08 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 11509 0 0 0 5107 1 0 0 25 0 1 0 599393204 59101184 11244 33554432000 4194304 4785740 140736552748448 18446744073709551615 4287050 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 14429 11244 340 145 0 11474 0
[pid=24575] ppid=24572 vsize=457412 CPUtime=25.52 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 997 0 0 0 2552 0 0 0 25 0 1 0 599393204 468389888 894 33554432000 134512640 134534564 4291023728 18446744073709551615 134528944 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 114353 894 95 6 0 113019 0
[pid=24576] ppid=24572 vsize=20060 CPUtime=51.08 cores=3

################
# More data... #
################

/proc/24577/statm: 3881 1737 175 34 0 2323 0
Current children cumulated CPU time (s) 204.06
Current children cumulated vsize (KiB) 1542372

[startup+102.307 s]
/proc/loadavg: 10.14 10.12 9.43 11/188 24612
/proc/meminfo: memFree=12376240/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=998892 CPUtime=102.3 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 39581 0 0 0 10225 5 0 0 25 0 1 0 599393204 1022865408 22500 33554432000 4194304 4587354 140734776412336 18446744073709551615 4364888 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 249723 22500 337 96 0 246285 0
[pid=24574] ppid=24572 vsize=66432 CPUtime=102.3 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 13510 0 0 0 10228 2 0 0 25 0 1 0 599393204 68026368 13245 33554432000 4194304 4785740 140736552748448 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 16608 13245 340 145 0 13653 0
[pid=24575] ppid=24572 vsize=457412 CPUtime=51.01 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 997 0 0 0 5101 0 0 0 25 0 1 0 599393204 468389888 894 33554432000 134512640 134534564 4291023728 18446744073709551615 134530586 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 114353 894 95 6 0 113019 0
[pid=24576] ppid=24572 vsize=26660 CPUtime=102.29 cores=3
/proc/24576/stat : 24576 (lingeling) R 24572 24572 24128 0 -1 4202496 28791 0 0 0 10225 4 0 0 25 0 1 0 599393204 27299840 5827 33554432000 4194304 4352236 140734812281360 18446744073709551615 4328351 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24576/statm: 6665 5827 127 39 0 5745 0
[pid=24577] ppid=24572 vsize=20152 CPUtime=50.72 cores=7
/proc/24577/stat : 24577 (march_hi) R 24572 24572 24128 0 -1 4202496 3954 0 0 0 5071 1 0 0 25 0 1 0 599393204 20635648 2497 33554432000 4194304 4331988 140734352815504 18446744073709551615 4228311 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24577/statm: 5038 2497 175 34 0 3480 0
Current children cumulated CPU time (s) 408.62
Current children cumulated vsize (KiB) 1587872

[startup+162.301 s]
/proc/loadavg: 10.23 10.15 9.48 11/188 24614
/proc/meminfo: memFree=12315212/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=7892664 CPUtime=162.29 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 51262 0 0 0 16221 8 0 0 25 0 1 0 599393204 8082087936 24851 33554432000 4194304 4587354 140734776412336 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 1973166 24851 337 96 0 1969728 0
[pid=24574] ppid=24572 vsize=89972 CPUtime=162.29 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 18873 0 0 0 16226 3 0 0 25 0 1 0 599393204 92131328 18608 33554432000 4194304 4785740 140736552748448 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 22493 18608 340 145 0 19538 0
[pid=24575] ppid=24572 vsize=457412 CPUtime=80.82 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 997 0 0 0 8082 0 0 0 25 0 1 0 599393204 468389888 894 33554432000 134512640 134534564 4291023728 18446744073709551615 134530538 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 114353 894 95 6 0 113019 0
[pid=24576] ppid=24572 vsize=35364 CPUtime=162.28 cores=3
/proc/24576/stat : 24576 (lingeling) R 24572 24572 24128 0 -1 4202496 42423 0 0 0 16223 5 0 0 25 0 1 0 599393204 36212736 8003 33554432000 4194304 4352236 140734812281360 18446744073709551615 4276626 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24576/statm: 8841 8003 127 39 0 7921 0
[pid=24577] ppid=24572 vsize=24348 CPUtime=80.58 cores=7
/proc/24577/stat : 24577 (march_hi) R 24572 24572 24128 0 -1 4202496 5221 0 0 0 8057 1 0 0 25 0 1 0 599393204 24932352 3764 33554432000 4194304 4331988 140734352815504 18446744073709551615 4297280 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24577/statm: 6087 3764 175 34 0 4529 0
Current children cumulated CPU time (s) 648.26
Current children cumulated vsize (KiB) 8518084

[startup+222.301 s]
/proc/loadavg: 10.17 10.15 9.52 11/188 24615
/proc/meminfo: memFree=12288040/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=7893924 CPUtime=222.29 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 80699 0 0 0 22213 16 0 0 25 0 1 0 599393204 8083378176 24452 33554432000 4194304 4587354 140734776412336 18446744073709551615 4239304 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 1973481 24452 337 96 0 1970043 0
[pid=24574] ppid=24572 vsize=97332 CPUtime=222.29 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 20620 0 0 0 22226 3 0 0 25 0 1 0 599393204 99667968 20355 33554432000 4194304 4785740 140736552748448 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 24333 20355 340 145 0 21378 0
[pid=24575] ppid=24572 vsize=457412 CPUtime=110.69 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 997 0 0 0 11069 0 0 0 25 0 1 0 599393204 468389888 894 33554432000 134512640 134534564 4291023728 18446744073709551615 134530126 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 114353 894 95 6 0 113019 0
[pid=24576] ppid=24572 vsize=35620 CPUtime=222.28 cores=3
/proc/24576/stat : 24576 (lingeling) R 24572 24572 24128 0 -1 4202496 59148 0 0 0 22221 7 0 0 25 0 1 0 599393204 36474880 8067 33554432000 4194304 4352236 140734812281360 18446744073709551615 4328345 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24576/statm: 8905 8067 127 39 0 7985 0
[pid=24577] ppid=24572 vsize=32060 CPUtime=110.38 cores=7
/proc/24577/stat : 24577 (march_hi) R 24572 24572 24128 0 -1 4202496 7027 0 0 0 11035 3 0 0 25 0 1 0 599393204 32829440 5570 33554432000 4194304 4331988 140734352815504 18446744073709551615 4228288 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24577/statm: 8015 5570 175 34 0 6457 0
Current children cumulated CPU time (s) 887.93
Current children cumulated vsize (KiB) 8534672

[startup+282.301 s]
/proc/loadavg: 10.29 10.18 9.57 11/188 24617
/proc/meminfo: memFree=12260616/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=7894916 CPUtime=282.3 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 100703 0 0 0 28208 22 0 0 25 0 1 0 599393204 8084393984 26203 33554432000 4194304 4587354 140734776412336 18446744073709551615 216734875334 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 1973729 26203 337 96 0 1970291 0
[pid=24574] ppid=24572 vsize=101600 CPUtime=282.29 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 21614 0 0 0 28226 3 0 0 25 0 1 0 599393204 104038400 21349 33554432000 4194304 4785740 140736552748448 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 25400 21349 340 145 0 22445 0
[pid=24575] ppid=24572 vsize=457412 CPUtime=140.5 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 997 0 0 0 14050 0 0 0 25 0 1 0 599393204 468389888 894 33554432000 134512640 134534564 4291023728 18446744073709551615 134528913 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 114353 894 95 6 0 113019 0
[pid=24576] ppid=24572 vsize=38184 CPUtime=282.29 cores=3
/proc/24576/stat : 24576 (lingeling) R 24572 24572 24128 0 -1 4202496 70349 0 0 0 28220 9 0 0 25 0 1 0 599393204 39100416 8708 33554432000 4194304 4352236 140734812281360 18446744073709551615 4328345 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24576/statm: 9546 8708 127 39 0 8626 0
[pid=24577] ppid=24572 vsize=40924 CPUtime=140.25 cores=7
/proc/24577/stat : 24577 (march_hi) R 24572 24572 24128 0 -1 4202496 8856 0 0 0 14021 4 0 0 25 0 1 0 599393204 41906176 7399 33554432000 4194304 4331988 140734352815504 18446744073709551615 4287256 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24577/statm: 10231 7399 175 34 0 8673 0
Current children cumulated CPU time (s) 1127.63
Current children cumulated vsize (KiB) 8551360

[startup+342.301 s]
/proc/loadavg: 10.23 10.18 9.60 11/188 24619
/proc/meminfo: memFree=12202820/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=1021980 CPUtime=342.3 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 122624 0 0 0 34201 29 0 0 25 0 1 0 599393204 1046507520 28153 33554432000 4194304 4587354 140734776412336 18446744073709551615 4238608 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 255495 28153 337 96 0 252057 0
[pid=24574] ppid=24572 vsize=103308 CPUtime=342.29 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 21982 0 0 0 34226 3 0 0 25 0 1 0 599393204 105787392 21717 33554432000 4194304 4785740 140736552748448 18446744073709551615 4287252 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 25827 21717 340 145 0 22872 0
[pid=24575] ppid=24572 vsize=457412 CPUtime=170.35 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 997 0 0 0 17035 0 0 0 25 0 1 0 599393204 468389888 894 33554432000 134512640 134534564 4291023728 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 114353 894 95 6 0 113019 0
[pid=24576] ppid=24572 vsize=44328 CPUtime=342.29 cores=3
/proc/24576/stat : 24576 (lingeling) R 24572 24572 24128 0 -1 4202496 78093 0 0 0 34219 10 0 0 25 0 1 0 599393204 45391872 10244 33554432000 4194304 4352236 140734812281360 18446744073709551615 4231009 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24576/statm: 11082 10244 127 39 0 10162 0
[pid=24577] ppid=24572 vsize=56540 CPUtime=170.07 cores=7
/proc/24577/stat : 24577 (march_hi) R 24572 24572 24128 0 -1 4202496 10546 0 0 0 17002 5 0 0 25 0 1 0 599393204 57896960 9089 33554432000 4194304 4331988 140734352815504 18446744073709551615 4316070 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24577/statm: 14135 9089 175 34 0 12577 0
Current children cumulated CPU time (s) 1367.3
Current children cumulated vsize (KiB) 1701892



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+369.001 s]
/proc/loadavg: 10.23 10.18 9.62 11/188 24619
/proc/meminfo: memFree=12189420/32951124 swapFree=67102140/67111528
[pid=24572] ppid=24570 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/24572/stat : 24572 (ppfolio) S 24570 24572 24128 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 599393203 18763776 389 33554432000 4194304 4287625 140737206771360 18446744073709551615 216735198735 0 0 4096 65536 4599372317124984831 0 0 17 3 0 0 0
/proc/24572/statm: 4581 389 341 23 0 91 0
[pid=24573] ppid=24572 vsize=7896108 CPUtime=369 cores=1
/proc/24573/stat : 24573 (cryptominisat) R 24572 24572 24128 0 -1 4202496 134246 0 0 0 36868 32 0 0 25 0 1 0 599393204 8085614592 25069 33554432000 4194304 4587354 140734776412336 18446744073709551615 4245761 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/24573/statm: 1974027 25069 337 96 0 1970589 0
[pid=24574] ppid=24572 vsize=104788 CPUtime=369 cores=5
/proc/24574/stat : 24574 (clasp) R 24572 24572 24128 0 -1 4202496 22283 0 0 0 36897 3 0 0 25 0 1 0 599393204 107302912 22018 33554432000 4194304 4785740 140736552748448 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/24574/statm: 26197 22018 340 145 0 23242 0
[pid=24575] ppid=24572 vsize=457412 CPUtime=183.61 cores=7
/proc/24575/stat : 24575 (TNM) R 24572 24572 24128 0 -1 4202496 997 0 0 0 18361 0 0 0 25 0 1 0 599393204 468389888 894 33554432000 134512640 134534564 4291023728 18446744073709551615 134528687 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24575/statm: 114353 894 95 6 0 113019 0
[pid=24576] ppid=24572 vsize=53800 CPUtime=369 cores=3
/proc/24576/stat : 24576 (lingeling) R 24572 24572 24128 0 -1 4202496 88077 0 0 0 36888 12 0 0 25 0 1 0 599393204 55091200 12612 33554432000 4194304 4352236 140734812281360 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/24576/statm: 13450 12612 127 39 0 12530 0
[pid=24577] ppid=24572 vsize=56540 CPUtime=183.37 cores=7
/proc/24577/stat : 24577 (march_hi) R 24572 24572 24128 0 -1 4202496 11297 0 0 0 18332 5 0 0 25 0 1 0 599393204 57896960 9840 33554432000 4194304 4331988 140734352815504 18446744073709551615 4226217 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/24577/statm: 14135 9840 175 34 0 12577 0
Current children cumulated CPU time (s) 1473.98
Current children cumulated vsize (KiB) 8586972

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 24572 and gives
#  childrusage.ru_utime.tv_sec=735
#  childrusage.ru_utime.tv_usec=827137
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=190970
# CPU time returned by wait4() is 736.018
# while last known CPU time is 1473.98
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 369.013
CPU time (s): 1473.98
CPU user time (s): 1473.46
CPU system time (s): 0.52
CPU usage (%): 399.438
Max. virtual memory (cumulated for all children) (KiB): 16376436

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 735.827
system time used= 0.19097
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 100893
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 306
involuntary context switches= 8645

runsolver used 0.552915 second user time and 1.46978 second system time

The end

Launcher Data 

Begin job on node119 at 2011-04-29 17:40:25
IDJOB=3263537
IDBENCH=83985
IDSOLVER=1612
FILE ID=node119/3263537-1304091625
RUNJOBID= node119-1304090423-24478
PBS_JOBID= 13170550
Free space on /tmp= 71804 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_70_15_90_1-dir.shuffled-as.sat03-136.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3263537-1304091625/watcher-3263537-1304091625 -o /tmp/evaluation-result-3263537-1304091625/solver-3263537-1304091625 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3263537-1304091625.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5bd7c76aed56f767985526812cdb8e0e
RANDOM SEED=1791534488

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      12741396 kB
Buffers:       2234872 kB
Cached:       17325848 kB
SwapCached:       5504 kB
Active:        7992320 kB
Inactive:     11631572 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      12741396 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:           10528 kB
Writeback:           0 kB
AnonPages:       60532 kB
Mapped:          15604 kB
Slab:           521652 kB
PageTables:       5056 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   222400 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71796 MiB
End job on node119 at 2011-04-29 17:46:36