Trace number 3265341

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.26 1201.71

General information on the benchmark

NameSAT05/crafted/sabharwal05/counting/fphp/unsat/
harder/fphp-020-019.shuffled-as.sat05-1232.cnf
MD5SUMabb32bff5eb954e790612a6209864b5d
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark2.08668
Satisfiable
(Un)Satisfiability was proved
Number of variables380
Number of clauses7050
Sum of the clauses size14440
Maximum clause length19
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 27030
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 520

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.00	Starting clasp 1.3.6 on core(s) 5
0.00/0.00	Starting TNM 2009 on core(s) 7
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.00	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3265341-1304095587/watcher-3265341-1304095587 -o /tmp/evaluation-result-3265341-1304095587/solver-3265341-1304095587 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3265341-1304095587.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.72 5.61 7.80 2/171 25033
/proc/meminfo: memFree=15222416/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) R 25031 25033 24128 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 0 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0

[startup+0.331062 s]
/proc/loadavg: 3.72 5.61 7.80 2/171 25033
/proc/meminfo: memFree=15222416/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 0 0 0 24 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.401051 s]
/proc/loadavg: 3.72 5.61 7.80 2/171 25033
/proc/meminfo: memFree=15222416/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 0 0 0 24 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.501032 s]
/proc/loadavg: 3.72 5.61 7.80 2/171 25033
/proc/meminfo: memFree=15222416/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 0 0 0 22 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700997 s]
/proc/loadavg: 3.72 5.61 7.80 2/171 25033
/proc/meminfo: memFree=15222416/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 0 0 0 20 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50086 s]
/proc/loadavg: 3.72 5.61 7.80 6/177 25039
/proc/meminfo: memFree=15189708/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=570488 CPUtime=1.49 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 7284 0 0 0 148 1 0 0 25 0 1 0 599789318 584179712 4847 33554432000 4194304 4587354 140734811591664 18446744073709551615 4365356 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 142622 4847 337 96 0 139184 0
[pid=25035] ppid=25033 vsize=17064 CPUtime=1.49 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 1727 0 0 0 149 0 0 0 25 0 1 0 599789318 17473536 1616 33554432000 4194304 4785740 140734448130704 18446744073709551615 4552626 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 4266 1616 340 145 0 1311 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=0.79 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 79 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134529896 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=11356 CPUtime=1.49 cores=3
/proc/25037/stat : 25037 (lingeling) R 25033 25033 24128 0 -1 4202496 3812 0 0 0 149 0 0 0 25 0 1 0 599789318 11628544 2033 33554432000 4194304 4352236 140735824930256 18446744073709551615 4207645 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25037/statm: 2839 2033 127 39 0 1919 0
[pid=25038] ppid=25033 vsize=8108 CPUtime=0.68 cores=7
/proc/25038/stat : 25038 (march_hi) R 25033 25033 24128 0 -1 4202496 625 0 0 0 68 0 0 0 25 0 1 0 599789318 8302592 446 33554432000 4194304 4331988 140736912027504 18446744073709551615 4226434 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25038/statm: 2027 446 174 34 0 469 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 1081436

[startup+3.10058 s]
/proc/loadavg: 3.72 5.61 7.80 6/177 25039
/proc/meminfo: memFree=15187972/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=571292 CPUtime=3.08 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 8317 0 0 0 307 1 0 0 25 0 1 0 599789318 585003008 4970 33554432000 4194304 4587354 140734811591664 18446744073709551615 4238571 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 142823 4970 337 96 0 139385 0
[pid=25035] ppid=25033 vsize=18644 CPUtime=3.09 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 2093 0 0 0 309 0 0 0 25 0 1 0 599789318 19091456 1982 33554432000 4194304 4785740 140734448130704 18446744073709551615 4287030 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 4661 1982 340 145 0 1706 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=1.59 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 159 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134528824 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=12380 CPUtime=3.09 cores=3
/proc/25037/stat : 25037 (lingeling) R 25033 25033 24128 0 -1 4202496 5732 0 0 0 309 0 0 0 25 0 1 0 599789318 12677120 2289 33554432000 4194304 4352236 140735824930256 18446744073709551615 4329086 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25037/statm: 3095 2289 127 39 0 2175 0
[pid=25038] ppid=25033 vsize=8312 CPUtime=1.48 cores=7
/proc/25038/stat : 25038 (march_hi) R 25033 25033 24128 0 -1 4202496 671 0 0 0 148 0 0 0 25 0 1 0 599789318 8511488 492 33554432000 4194304 4331988 140736912027504 18446744073709551615 4226263 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25038/statm: 2078 492 175 34 0 520 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 1085048

[startup+6.30103 s]
/proc/loadavg: 3.82 5.60 7.78 6/177 25039
/proc/meminfo: memFree=15180036/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=574324 CPUtime=6.29 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 12325 0 0 0 627 2 0 0 25 0 1 0 599789318 588107776 5748 33554432000 4194304 4587354 140734811591664 18446744073709551615 4364788 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 143581 5748 337 96 0 140143 0
[pid=25035] ppid=25033 vsize=20772 CPUtime=6.29 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 2582 0 0 0 629 0 0 0 25 0 1 0 599789318 21270528 2471 33554432000 4194304 4785740 140734448130704 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 5193 2471 340 145 0 2238 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=3.19 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 319 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134529082 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=18012 CPUtime=6.29 cores=3
/proc/25037/stat : 25037 (lingeling) R 25033 25033 24128 0 -1 4202496 9188 0 0 0 628 1 0 0 25 0 1 0 599789318 18444288 3697 33554432000 4194304 4352236 140735824930256 18446744073709551615 4276530 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25037/statm: 4503 3697 127 39 0 3583 0
[pid=25038] ppid=25033 vsize=8564 CPUtime=3.06 cores=7
/proc/25038/stat : 25038 (march_hi) R 25033 25033 24128 0 -1 4202496 725 0 0 0 306 0 0 0 25 0 1 0 599789318 8769536 546 33554432000 4194304 4331988 140736912027504 18446744073709551615 4237192 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25038/statm: 2141 546 175 34 0 583 0
Current children cumulated CPU time (s) 25.12
Current children cumulated vsize (KiB) 1096092

[startup+12.7009 s]
/proc/loadavg: 3.92 5.59 7.77 6/177 25040
/proc/meminfo: memFree=15152824/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=592800 CPUtime=12.69 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 19940 0 0 0 1266 3 0 0 25 0 1 0 599789318 607027200 10694 33554432000 4194304 4587354 140734811591664 18446744073709551615 4365356 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 148200 10694 337 96 0 144762 0
[pid=25035] ppid=25033 vsize=22848 CPUtime=12.69 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 3094 0 0 0 1269 0 0 0 25 0 1 0 599789318 23396352 2983 33554432000 4194304 4785740 140734448130704 18446744073709551615 4287019 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 5712 2983 340 145 0 2757 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=6.39 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 639 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134529640 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=19196 CPUtime=12.69 cores=3
/proc/25037/stat : 25037 (lingeling) R 25033 25033 24128 0 -1 4202496 15860 0 0 0 1268 1 0 0 25 0 1 0 599789318 19656704 3969 33554432000 4194304 4352236 140735824930256 18446744073709551615 4327896 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25037/statm: 4799 3969 127 39 0 3879 0
[pid=25038] ppid=25033 vsize=9132 CPUtime=6.23 cores=7
/proc/25038/stat : 25038 (march_hi) R 25033 25033 24128 0 -1 4202496 849 0 0 0 623 0 0 0 25 0 1 0 599789318 9351168 670 33554432000 4194304 4331988 140736912027504 18446744073709551615 4265456 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25038/statm: 2283 670 175 34 0 725 0
Current children cumulated CPU time (s) 50.69
Current children cumulated vsize (KiB) 1118396

[startup+25.5007 s]
/proc/loadavg: 4.24 5.57 7.73 6/177 25040
/proc/meminfo: memFree=15139852/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=595988 CPUtime=25.5 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 28093 0 0 0 2545 5 0 0 25 0 1 0 599789318 610291712 11525 33554432000 4194304 4587354 140734811591664 18446744073709551615 4238568 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 148997 11526 337 96 0 145559 0
[pid=25035] ppid=25033 vsize=26260 CPUtime=25.49 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 3885 0 0 0 2549 0 0 0 25 0 1 0 599789318 26890240 3774 33554432000 4194304 4785740 140734448130704 18446744073709551615 4287050 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 6565 3774 341 145 0 3610 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=12.73 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 1273 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134530571 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=27392 CPUtime=25.49 cores=3
/proc/25037/stat : 25037 (lingeling) R 25033 25033 24128 0 -1 4202496 31237 0 0 0 2546 3 0 0 25 0 1 0 599789318 28049408 6034 33554432000 4194304 4352236 140735824930256 18446744073709551615 4328832 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25037/statm: 6848 6034 127 39 0 5928 0
[pid=25038] ppid=25033 vsize=10408 CPUtime=12.62 cores=7
/proc/25038/stat : 25038 (march_hi) R 25033 25033 24128 0 -1 4202496 1082 0 0 0 1262 0 0 0 25 0 1 0 599789318 10657792 903 33554432000 4194304 4331988 140736912027504 18446744073709551615 4316666 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25038/statm: 2602 903 175 34 0 1044 0
Current children cumulated CPU time (s) 101.83
Current children cumulated vsize (KiB) 1134468

[startup+51.1012 s]
/proc/loadavg: 4.56 5.54 7.66 6/177 25041
/proc/meminfo: memFree=15091064/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=630120 CPUtime=51.09 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 49010 0 0 0 5101 8 0 0 25 0 1 0 599789318 645242880 21412 33554432000 4194304 4587354 140734811591664 18446744073709551615 4365216 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 157530 21412 337 96 0 154092 0
[pid=25035] ppid=25033 vsize=31056 CPUtime=51.08 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 4955 0 0 0 5108 0 0 0 25 0 1 0 599789318 31801344 4844 33554432000 4194304 4785740 140734448130704 18446744073709551615 4553344 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 7764 4844 341 145 0 4809 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=25.49 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 2549 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134530701 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=30464 CPUtime=51.08 cores=3

################
# More data... #
################

/proc/25038/statm: 24194 19105 175 34 0 22636 0
Current children cumulated CPU time (s) 3524.39
Current children cumulated vsize (KiB) 1508488

[startup+942.301 s]
/proc/loadavg: 5.09 5.09 6.05 6/177 25073
/proc/meminfo: memFree=14772596/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0.01 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 1 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=799004 CPUtime=942.27 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 410606 0 0 0 94140 87 0 0 25 0 1 0 599789318 818180096 64567 33554432000 4194304 4587354 140734811591664 18446744073709551615 4237681 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 199751 64567 337 96 0 196313 0
[pid=25035] ppid=25033 vsize=63548 CPUtime=942.29 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 12238 0 0 0 94227 2 0 0 25 0 1 0 599789318 65073152 12127 33554432000 4194304 4785740 140734448130704 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 15887 12127 341 145 0 12932 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=468.63 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 46863 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134529640 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=72104 CPUtime=942.31 cores=3
/proc/25037/stat : 25037 (lingeling) R 25033 25033 24128 0 -1 4202496 417567 0 0 0 94179 52 0 0 25 0 1 0 599789318 73834496 17196 33554432000 4194304 4352236 140735824930256 18446744073709551615 4281167 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25037/statm: 18026 17196 127 39 0 17106 0
[pid=25038] ppid=25033 vsize=96776 CPUtime=468.55 cores=7
/proc/25038/stat : 25038 (march_hi) R 25033 25033 24128 0 -1 4202496 20472 0 0 0 46845 10 0 0 25 0 1 0 599789318 99098624 20293 33554432000 4194304 4331988 140736912027504 18446744073709551615 4226499 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25038/statm: 24194 20293 175 34 0 22636 0
Current children cumulated CPU time (s) 3764.06
Current children cumulated vsize (KiB) 1505852

[startup+1002.3 s]
/proc/loadavg: 5.09 5.09 5.98 6/177 25075
/proc/meminfo: memFree=14712188/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0.01 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 1 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=798216 CPUtime=1002.28 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 441758 0 0 0 100133 95 0 0 25 0 1 0 599789318 817373184 69769 33554432000 4194304 4587354 140734811591664 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 199554 69769 337 96 0 196116 0
[pid=25035] ppid=25033 vsize=64056 CPUtime=1002.3 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 12374 0 0 0 100228 2 0 0 25 0 1 0 599789318 65593344 12263 33554432000 4194304 4785740 140734448130704 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 16014 12263 341 145 0 13059 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=498.51 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 49851 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134528396 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=104872 CPUtime=1002.31 cores=3
/proc/25037/stat : 25037 (lingeling) R 25033 25033 24128 0 -1 4202496 444191 0 0 0 100176 55 0 0 25 0 1 0 599789318 107388928 25388 33554432000 4194304 4352236 140735824930256 18446744073709551615 4331615 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25037/statm: 26218 25388 127 39 0 25298 0
[pid=25038] ppid=25033 vsize=96776 CPUtime=498.34 cores=7
/proc/25038/stat : 25038 (march_hi) R 25033 25033 24128 0 -1 4202496 21743 0 0 0 49824 10 0 0 25 0 1 0 599789318 99098624 21564 33554432000 4194304 4331988 140736912027504 18446744073709551615 4226237 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25038/statm: 24194 21564 175 34 0 22636 0
Current children cumulated CPU time (s) 4003.75
Current children cumulated vsize (KiB) 1538340

[startup+1062.3 s]
/proc/loadavg: 5.08 5.09 5.92 6/177 25076
/proc/meminfo: memFree=14730272/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0.01 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 1 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=798344 CPUtime=1062.27 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 469544 0 0 0 106125 102 0 0 25 0 1 0 599789318 817504256 63408 33554432000 4194304 4587354 140734811591664 18446744073709551615 4238763 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 199586 63408 337 96 0 196148 0
[pid=25035] ppid=25033 vsize=64248 CPUtime=1062.3 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 12469 0 0 0 106228 2 0 0 25 0 1 0 599789318 65789952 12358 33554432000 4194304 4785740 140734448130704 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 16062 12358 341 145 0 13107 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=528.31 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 52831 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134528934 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=105384 CPUtime=1062.31 cores=3
/proc/25037/stat : 25037 (lingeling) R 25033 25033 24128 0 -1 4202496 460703 0 0 0 106174 57 0 0 25 0 1 0 599789318 107913216 25516 33554432000 4194304 4352236 140735824930256 18446744073709551615 4328390 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25037/statm: 26346 25516 127 39 0 25426 0
[pid=25038] ppid=25033 vsize=141264 CPUtime=528.2 cores=7
/proc/25038/stat : 25038 (march_hi) R 25033 25033 24128 0 -1 4202496 23252 0 0 0 52809 11 0 0 25 0 1 0 599789318 144654336 23057 33554432000 4194304 4331988 140736912027504 18446744073709551615 4226530 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25038/statm: 35316 23057 175 34 0 33758 0
Current children cumulated CPU time (s) 4243.4
Current children cumulated vsize (KiB) 1583660

[startup+1122.31 s]
/proc/loadavg: 5.08 5.08 5.87 6/177 25078
/proc/meminfo: memFree=14714508/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0.01 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 1 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=999844 CPUtime=1122.28 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 499341 0 0 0 112119 109 0 0 25 0 1 0 599789318 1023840256 65019 33554432000 4194304 4587354 140734811591664 18446744073709551615 4243397 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 249961 65019 337 96 0 246523 0
[pid=25035] ppid=25033 vsize=66268 CPUtime=1122.3 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 12927 0 0 0 112228 2 0 0 25 0 1 0 599789318 67858432 12816 33554432000 4194304 4785740 140734448130704 18446744073709551615 4287038 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 16567 12816 341 145 0 13612 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=558.17 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 55817 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134529080 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=109480 CPUtime=1122.31 cores=3
/proc/25037/stat : 25037 (lingeling) R 25033 25033 24128 0 -1 4202496 470943 0 0 0 112173 58 0 0 25 0 1 0 599789318 112107520 26540 33554432000 4194304 4352236 140735824930256 18446744073709551615 4328090 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25037/statm: 27370 26540 127 39 0 26450 0
[pid=25038] ppid=25033 vsize=141264 CPUtime=558.03 cores=7
/proc/25038/stat : 25038 (march_hi) R 25033 25033 24128 0 -1 4202496 24368 0 0 0 55791 12 0 0 25 0 1 0 599789318 144654336 24173 33554432000 4194304 4331988 140736912027504 18446744073709551615 4300319 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25038/statm: 35316 24173 175 34 0 33758 0
Current children cumulated CPU time (s) 4483.1
Current children cumulated vsize (KiB) 1791276

[startup+1182.3 s]
/proc/loadavg: 5.07 5.08 5.82 6/177 25080
/proc/meminfo: memFree=14714616/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0.01 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 1 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=798344 CPUtime=1182.27 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 529124 0 0 0 118109 118 0 0 25 0 1 0 599789318 817504256 65943 33554432000 4194304 4587354 140734811591664 18446744073709551615 4237695 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 199586 65943 337 96 0 196148 0
[pid=25035] ppid=25033 vsize=66968 CPUtime=1182.3 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 13107 0 0 0 118228 2 0 0 25 0 1 0 599789318 68575232 12996 33554432000 4194304 4785740 140734448130704 18446744073709551615 4277703 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 16742 12996 341 145 0 13787 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=588 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 58800 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134530669 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=101288 CPUtime=1182.32 cores=3
/proc/25037/stat : 25037 (lingeling) R 25033 25033 24128 0 -1 4202496 486303 0 0 0 118171 61 0 0 25 0 1 0 599789318 103718912 24492 33554432000 4194304 4352236 140735824930256 18446744073709551615 4328004 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25037/statm: 25322 24492 127 39 0 24402 0
[pid=25038] ppid=25033 vsize=141264 CPUtime=587.86 cores=7
/proc/25038/stat : 25038 (march_hi) R 25033 25033 24128 0 -1 4202496 25577 0 0 0 58774 12 0 0 25 0 1 0 599789318 144654336 25382 33554432000 4194304 4331988 140736912027504 18446744073709551615 4226479 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25038/statm: 35316 25382 175 34 0 33758 0
Current children cumulated CPU time (s) 4722.76
Current children cumulated vsize (KiB) 1582284



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 5.13 5.09 5.80 6/177 25080
/proc/meminfo: memFree=14690676/32951124 swapFree=67102140/67111528
[pid=25033] ppid=25031 vsize=18324 CPUtime=0.01 cores=1,3,5,7
/proc/25033/stat : 25033 (ppfolio) S 25031 25033 24128 0 -1 4202496 524 0 0 0 0 1 0 0 15 0 1 0 599789317 18763776 390 33554432000 4194304 4287625 140735853754400 18446744073709551615 216735198735 0 0 4096 65536 12294950956952453119 0 0 17 5 0 0 0
/proc/25033/statm: 4581 390 341 23 0 91 0
[pid=25034] ppid=25033 vsize=798864 CPUtime=1201.68 cores=1
/proc/25034/stat : 25034 (cryptominisat) R 25033 25033 24128 0 -1 4202496 539481 0 0 0 120047 121 0 0 25 0 1 0 599789318 818036736 68803 33554432000 4194304 4587354 140734811591664 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/25034/statm: 199716 68803 337 96 0 196278 0
[pid=25035] ppid=25033 vsize=67980 CPUtime=1201.7 cores=5
/proc/25035/stat : 25035 (clasp) R 25033 25033 24128 0 -1 4202496 13308 0 0 0 120168 2 0 0 25 0 1 0 599789318 69611520 13197 33554432000 4194304 4785740 140734448130704 18446744073709551615 4287008 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/25035/statm: 16995 13197 341 145 0 14040 0
[pid=25036] ppid=25033 vsize=456096 CPUtime=597.64 cores=7
/proc/25036/stat : 25036 (TNM) R 25033 25033 24128 0 -1 4202496 430 0 0 0 59764 0 0 0 25 0 1 0 599789318 467042304 327 33554432000 134512640 134534564 4292941328 18446744073709551615 134528371 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25036/statm: 114024 327 95 6 0 112690 0
[pid=25037] ppid=25033 vsize=109480 CPUtime=1201.71 cores=3
/proc/25037/stat : 25037 (lingeling) R 25033 25033 24128 0 -1 4202496 492447 0 0 0 120110 61 0 0 25 0 1 0 599789318 112107520 26540 33554432000 4194304 4352236 140735824930256 18446744073709551615 4328035 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/25037/statm: 27370 26540 127 39 0 26450 0
[pid=25038] ppid=25033 vsize=141264 CPUtime=597.52 cores=7
/proc/25038/stat : 25038 (march_hi) R 25033 25033 24128 0 -1 4202496 25977 0 0 0 59740 12 0 0 25 0 1 0 599789318 144654336 25782 33554432000 4194304 4331988 140736912027504 18446744073709551615 4226263 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/25038/statm: 35316 25782 175 34 0 33758 0
Current children cumulated CPU time (s) 4800.26
Current children cumulated vsize (KiB) 1592008

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25033 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=2999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=13997
# CPU time returned by wait4() is 0.016996
# while last known CPU time is 4800.26
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.26
CPU user time (s): 4798.29
CPU system time (s): 1.97
CPU usage (%): 399.451
Max. virtual memory (cumulated for all children) (KiB): 2348212

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.002999
system time used= 0.013997
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 524
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 390
involuntary context switches= 6

runsolver used 1.76873 second user time and 4.78727 second system time

The end

Launcher Data 

Begin job on node119 at 2011-04-29 18:46:27
IDJOB=3265341
IDBENCH=85542
IDSOLVER=1612
FILE ID=node119/3265341-1304095587
RUNJOBID= node119-1304090423-24478
PBS_JOBID= 13170550
Free space on /tmp= 71816 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT05/crafted/sabharwal05/counting/fphp/unsat/harder/fphp-020-019.shuffled-as.sat05-1232.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3265341-1304095587/watcher-3265341-1304095587 -o /tmp/evaluation-result-3265341-1304095587/solver-3265341-1304095587 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3265341-1304095587.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= abb32bff5eb954e790612a6209864b5d
RANDOM SEED=1984863767

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15222696 kB
Buffers:       2210260 kB
Cached:       14867988 kB
SwapCached:       5504 kB
Active:        4673804 kB
Inactive:     12465520 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15222696 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:           10120 kB
Writeback:           0 kB
AnonPages:       58556 kB
Mapped:          14564 kB
Slab:           520892 kB
PageTables:       4452 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   189620 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71808 MiB
End job on node119 at 2011-04-29 19:06:30