Trace number 3291422

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.32 1201.76

General information on the benchmark

NameSAT03/industrial/goldberg/hard_eq_check/
k2mul.miter.shuffled-as.sat03-355.cnf
MD5SUM77e0700cbaeeb3d63a4820608fc84761
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark45.3381
Satisfiable
(Un)Satisfiability was proved
Number of variables11680
Number of clauses74581
Sum of the clauses size252050
Maximum clause length86
Minimum clause length1
Number of clauses of size 14
Number of clauses of size 217176
Number of clauses of size 334696
Number of clauses of size 40
Number of clauses of size 522704
Number of clauses of size over 51

Solver Data 

0.00/0.80	This is pico-portfolio, a naive parallel portfolio
0.00/0.80	Copyright (C) 2011 Olivier Roussel
0.00/0.80	Using 4 processing units
0.00/0.80	This portfolio uses the following solvers:
0.00/0.80	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.80	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.80	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.80	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.80	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.80	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.80	Starting clasp 1.3.6 on core(s) 5
0.00/0.80	Starting TNM 2009 on core(s) 7
0.00/0.80	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.80	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3291422-1304092587/watcher-3291422-1304092587 -o /tmp/evaluation-result-3291422-1304092587/solver-3291422-1304092587 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3291422-1304092587.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.19 5.22 6.41 1/175 2258
/proc/meminfo: memFree=30816976/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18320 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) R 2256 2258 379 0 -1 4194304 419 0 0 0 0 0 0 0 25 0 1 0 520856585 18759680 355 33554432000 4194304 4287625 140734358179968 18446744073709551615 4266848 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/2258/statm: 4580 360 313 23 0 90 0

[startup+0.267261 s]
/proc/loadavg: 5.19 5.22 6.41 1/175 2258
/proc/meminfo: memFree=30816976/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300257 s]
/proc/loadavg: 5.19 5.22 6.41 1/175 2258
/proc/meminfo: memFree=30816976/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.40024 s]
/proc/loadavg: 5.19 5.22 6.41 1/175 2258
/proc/meminfo: memFree=30816976/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700184 s]
/proc/loadavg: 5.19 5.22 6.41 1/175 2258
/proc/meminfo: memFree=30816976/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50103 s]
/proc/loadavg: 5.19 5.22 6.41 6/181 2264
/proc/meminfo: memFree=30743768/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=30272 CPUtime=1.49 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 2618 0 0 0 149 0 0 0 20 0 1 0 520856586 30998528 2449 33554432000 4194304 4587354 140733633142512 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 7568 2449 306 96 0 4130 0
[pid=2260] ppid=2258 vsize=22424 CPUtime=1.49 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 3617 0 0 0 149 0 0 0 25 0 1 0 520856586 22962176 2922 33554432000 4194304 4785740 140735221116336 18446744073709551615 4287204 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 5606 2922 340 145 0 2651 0
[pid=2261] ppid=2258 vsize=457276 CPUtime=0.74 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 1960 0 0 0 74 0 0 0 25 0 1 0 520856586 468250624 1857 33554432000 134512640 134534564 4292978880 18446744073709551615 134524483 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114319 1857 91 6 0 112985 0
[pid=2262] ppid=2258 vsize=11932 CPUtime=1.49 cores=3
/proc/2262/stat : 2262 (lingeling) R 2258 2258 379 0 -1 4202496 3062 0 0 0 149 0 0 0 25 0 1 0 520856586 12218368 2150 33554432000 4194304 4352236 140736224321184 18446744073709551615 4303846 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2262/statm: 2983 2150 125 39 0 2063 0
[pid=2263] ppid=2258 vsize=76112 CPUtime=0.73 cores=7
/proc/2263/stat : 2263 (march_hi) R 2258 2258 379 0 -1 4202496 23542 0 0 0 70 3 0 0 25 0 1 0 520856586 77938688 13845 33554432000 4194304 4331988 140733772606736 18446744073709551615 4271184 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2263/statm: 19028 13845 151 34 0 17470 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 616340

[startup+3.10074 s]
/proc/loadavg: 5.18 5.22 6.40 6/181 2264
/proc/meminfo: memFree=30655356/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=42304 CPUtime=3.09 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 6160 0 0 0 309 0 0 0 25 0 1 0 520856586 43319296 5169 33554432000 4194304 4587354 140733633142512 18446744073709551615 4250346 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 10576 5169 329 96 0 7138 0
[pid=2260] ppid=2258 vsize=23752 CPUtime=3.09 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 3943 0 0 0 309 0 0 0 25 0 1 0 520856586 24322048 3248 33554432000 4194304 4785740 140735221116336 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 5938 3248 340 145 0 2983 0
[pid=2261] ppid=2258 vsize=457276 CPUtime=1.54 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 1960 0 0 0 154 0 0 0 25 0 1 0 520856586 468250624 1857 33554432000 134512640 134534564 4292978880 18446744073709551615 134524582 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114319 1857 91 6 0 112985 0
[pid=2262] ppid=2258 vsize=11932 CPUtime=3.09 cores=3
/proc/2262/stat : 2262 (lingeling) R 2258 2258 379 0 -1 4202496 3062 0 0 0 309 0 0 0 25 0 1 0 520856586 12218368 2150 33554432000 4194304 4352236 140736224321184 18446744073709551615 4267912 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2262/statm: 2983 2150 125 39 0 2063 0
[pid=2263] ppid=2258 vsize=191636 CPUtime=1.53 cores=7
/proc/2263/stat : 2263 (march_hi) R 2258 2258 379 0 -1 4202496 54811 0 0 0 145 8 0 0 25 0 1 0 520856586 196235264 45114 33554432000 4194304 4331988 140733772606736 18446744073709551615 4226625 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2263/statm: 47909 45114 151 34 0 46351 0
Current children cumulated CPU time (s) 12.34
Current children cumulated vsize (KiB) 745224

[startup+6.30116 s]
/proc/loadavg: 5.18 5.22 6.40 6/181 2264
/proc/meminfo: memFree=30377792/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=42304 CPUtime=6.28 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 6469 0 0 0 628 0 0 0 25 0 1 0 520856586 43319296 5463 33554432000 4194304 4587354 140733633142512 18446744073709551615 4237665 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 10576 5463 331 96 0 7138 0
[pid=2260] ppid=2258 vsize=27992 CPUtime=6.28 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 4985 0 0 0 628 0 0 0 25 0 1 0 520856586 28663808 4290 33554432000 4194304 4785740 140735221116336 18446744073709551615 4553046 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 6998 4290 340 145 0 4043 0
[pid=2261] ppid=2258 vsize=458104 CPUtime=3.14 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 2757 0 0 0 314 0 0 0 25 0 1 0 520856586 469098496 2654 33554432000 134512640 134534564 4292978880 18446744073709551615 134528661 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114526 2654 95 6 0 113192 0
[pid=2262] ppid=2258 vsize=11932 CPUtime=6.29 cores=3
/proc/2262/stat : 2262 (lingeling) R 2258 2258 379 0 -1 4202496 3064 0 0 0 629 0 0 0 25 0 1 0 520856586 12218368 2152 33554432000 4194304 4352236 140736224321184 18446744073709551615 4327971 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2262/statm: 2983 2152 127 39 0 2063 0
[pid=2263] ppid=2258 vsize=531812 CPUtime=3.11 cores=7
/proc/2263/stat : 2263 (march_hi) R 2258 2258 379 0 -1 4202496 121021 0 0 0 294 17 0 0 25 0 1 0 520856586 544575488 111324 33554432000 4194304 4331988 140733772606736 18446744073709551615 4226645 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2263/statm: 132953 111324 151 34 0 131395 0
Current children cumulated CPU time (s) 25.1
Current children cumulated vsize (KiB) 1090468

[startup+12.701 s]
/proc/loadavg: 5.15 5.21 6.38 6/181 2264
/proc/meminfo: memFree=30268100/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=57356 CPUtime=12.68 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 10355 0 0 0 1267 1 0 0 25 0 1 0 520856586 58732544 8388 33554432000 4194304 4587354 140733633142512 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 14339 8388 338 96 0 10901 0
[pid=2260] ppid=2258 vsize=31728 CPUtime=12.69 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 5949 0 0 0 1268 1 0 0 25 0 1 0 520856586 32489472 5254 33554432000 4194304 4785740 140735221116336 18446744073709551615 4287252 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 7932 5254 340 145 0 4977 0
[pid=2261] ppid=2258 vsize=458104 CPUtime=6.31 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 2757 0 0 0 631 0 0 0 25 0 1 0 520856586 469098496 2654 33554432000 134512640 134534564 4292978880 18446744073709551615 134530568 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114526 2654 95 6 0 113192 0
[pid=2262] ppid=2258 vsize=12920 CPUtime=12.69 cores=3
/proc/2262/stat : 2262 (lingeling) R 2258 2258 379 0 -1 4202496 3443 0 0 0 1269 0 0 0 25 0 1 0 520856586 13230080 2399 33554432000 4194304 4352236 140736224321184 18446744073709551615 4278746 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2262/statm: 3230 2399 127 39 0 2310 0
[pid=2263] ppid=2258 vsize=578124 CPUtime=6.3 cores=7
/proc/2263/stat : 2263 (march_hi) R 2258 2258 379 0 -1 4202496 235925 0 0 0 599 31 0 0 25 0 1 0 520856586 591998976 123497 33554432000 4194304 4331988 140733772606736 18446744073709551615 271549932969 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2263/statm: 144531 123497 151 34 0 142973 0
Current children cumulated CPU time (s) 50.67
Current children cumulated vsize (KiB) 1156556

[startup+25.5067 s]
/proc/loadavg: 5.12 5.20 6.37 6/181 2264
/proc/meminfo: memFree=30233264/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=156548 CPUtime=25.5 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 11697 0 0 0 2547 3 0 0 25 0 1 0 520856586 160305152 8442 33554432000 4194304 4587354 140733633142512 18446744073709551615 4238766 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 39137 8442 338 96 0 35699 0
[pid=2260] ppid=2258 vsize=44964 CPUtime=25.5 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 9205 0 0 0 2547 3 0 0 25 0 1 0 520856586 46043136 8510 33554432000 4194304 4785740 140735221116336 18446744073709551615 4286953 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 11241 8510 340 145 0 8286 0
[pid=2261] ppid=2258 vsize=458104 CPUtime=12.65 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 2757 0 0 0 1265 0 0 0 25 0 1 0 520856586 469098496 2654 33554432000 134512640 134534564 4292978880 18446744073709551615 134530379 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114526 2654 95 6 0 113192 0
[pid=2262] ppid=2258 vsize=16200 CPUtime=25.49 cores=3
/proc/2262/stat : 2262 (lingeling) R 2258 2258 379 0 -1 4202496 4263 0 0 0 2549 0 0 0 25 0 1 0 520856586 16588800 3219 33554432000 4194304 4352236 140736224321184 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2262/statm: 4050 3219 127 39 0 3130 0
[pid=2263] ppid=2258 vsize=689132 CPUtime=12.7 cores=7
/proc/2263/stat : 2263 (march_hi) R 2258 2258 379 0 -1 4202496 322960 0 0 0 1226 44 0 0 25 0 1 0 520856586 705671168 128294 33554432000 4194304 4331988 140733772606736 18446744073709551615 4252204 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2263/statm: 172283 128294 151 34 0 170725 0
Current children cumulated CPU time (s) 101.84
Current children cumulated vsize (KiB) 1383272

[startup+51.101 s]
/proc/loadavg: 5.08 5.18 6.33 6/181 2265
/proc/meminfo: memFree=29664816/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=162512 CPUtime=51.09 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 15897 0 0 0 5105 4 0 0 25 0 1 0 520856586 166412288 9723 33554432000 4194304 4587354 140733633142512 18446744073709551615 4248785 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 40628 9723 338 96 0 37190 0
[pid=2260] ppid=2258 vsize=63600 CPUtime=51.09 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 13886 0 0 0 5104 5 0 0 25 0 1 0 520856586 65126400 13191 33554432000 4194304 4785740 140735221116336 18446744073709551615 4553423 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 15900 13191 340 145 0 12945 0
[pid=2261] ppid=2258 vsize=458104 CPUtime=25.41 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 2757 0 0 0 2541 0 0 0 25 0 1 0 520856586 469098496 2654 33554432000 134512640 134534564 4292978880 18446744073709551615 134530712 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114526 2654 95 6 0 113192 0
[pid=2262] ppid=2258 vsize=25872 CPUtime=51.09 cores=3

################
# More data... #
################

/proc/2263/statm: 352680 262041 175 34 0 351122 0
Current children cumulated CPU time (s) 3764.12
Current children cumulated vsize (KiB) 2309764

[startup+1002.31 s]
/proc/loadavg: 5.06 5.10 5.52 6/181 2302
/proc/meminfo: memFree=29476348/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=187604 CPUtime=1002.32 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 71006 0 0 0 100200 32 0 0 25 0 1 0 520856586 192106496 19436 33554432000 4194304 4587354 140733633142512 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 46901 19436 338 96 0 43463 0
[pid=2260] ppid=2258 vsize=163900 CPUtime=1002.32 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 38814 0 0 0 100215 17 0 0 25 0 1 0 520856586 167833600 38119 33554432000 4194304 4785740 140735221116336 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 40975 38119 340 145 0 38020 0
[pid=2261] ppid=2258 vsize=458104 CPUtime=498.42 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 2757 0 0 0 49842 0 0 0 25 0 1 0 520856586 469098496 2654 33554432000 134512640 134534564 4292978880 18446744073709551615 134528993 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114526 2654 95 6 0 113192 0
[pid=2262] ppid=2258 vsize=54096 CPUtime=1002.31 cores=3
/proc/2262/stat : 2262 (lingeling) R 2258 2258 379 0 -1 4202496 99625 0 0 0 100217 14 0 0 25 0 1 0 520856586 55394304 12693 33554432000 4194304 4352236 140736224321184 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2262/statm: 13524 12693 127 39 0 12604 0
[pid=2263] ppid=2258 vsize=1410720 CPUtime=498.43 cores=7
/proc/2263/stat : 2263 (march_hi) R 2258 2258 379 0 -1 4202496 785459 0 0 0 49734 109 0 0 25 0 1 0 520856586 1444577280 262088 33554432000 4194304 4331988 140733772606736 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2263/statm: 352680 262088 175 34 0 351122 0
Current children cumulated CPU time (s) 4003.8
Current children cumulated vsize (KiB) 2292748

[startup+1062.3 s]
/proc/loadavg: 5.05 5.09 5.49 6/181 2304
/proc/meminfo: memFree=29473336/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=195456 CPUtime=1062.32 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 75384 0 0 0 106198 34 0 0 25 0 1 0 520856586 200146944 20192 33554432000 4194304 4587354 140733633142512 18446744073709551615 4244125 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 48864 20193 338 96 0 45426 0
[pid=2260] ppid=2258 vsize=163900 CPUtime=1062.32 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 38821 0 0 0 106215 17 0 0 25 0 1 0 520856586 167833600 38126 33554432000 4194304 4785740 140735221116336 18446744073709551615 4278573 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 40975 38126 340 145 0 38020 0
[pid=2261] ppid=2258 vsize=458104 CPUtime=528.26 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 2757 0 0 0 52826 0 0 0 25 0 1 0 520856586 469098496 2654 33554432000 134512640 134534564 4292978880 18446744073709551615 134528412 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114526 2654 95 6 0 113192 0
[pid=2262] ppid=2258 vsize=54096 CPUtime=1062.3 cores=3
/proc/2262/stat : 2262 (lingeling) R 2258 2258 379 0 -1 4202496 106025 0 0 0 106216 14 0 0 25 0 1 0 520856586 55394304 12693 33554432000 4194304 4352236 140736224321184 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2262/statm: 13524 12693 127 39 0 12604 0
[pid=2263] ppid=2258 vsize=1410720 CPUtime=528.26 cores=7
/proc/2263/stat : 2263 (march_hi) R 2258 2258 379 0 -1 4202496 785514 0 0 0 52717 109 0 0 25 0 1 0 520856586 1444577280 262143 33554432000 4194304 4331988 140733772606736 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2263/statm: 352680 262143 175 34 0 351122 0
Current children cumulated CPU time (s) 4243.46
Current children cumulated vsize (KiB) 2300600

[startup+1122.3 s]
/proc/loadavg: 5.10 5.09 5.46 6/181 2306
/proc/meminfo: memFree=29479504/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=186920 CPUtime=1122.32 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 81039 0 0 0 112196 36 0 0 25 0 1 0 520856586 191406080 18773 33554432000 4194304 4587354 140733633142512 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 46730 18773 338 96 0 43292 0
[pid=2260] ppid=2258 vsize=163900 CPUtime=1122.32 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 38830 0 0 0 112215 17 0 0 25 0 1 0 520856586 167833600 38135 33554432000 4194304 4785740 140735221116336 18446744073709551615 4552983 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 40975 38135 340 145 0 38020 0
[pid=2261] ppid=2258 vsize=458104 CPUtime=558.07 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 2757 0 0 0 55807 0 0 0 25 0 1 0 520856586 469098496 2654 33554432000 134512640 134534564 4292978880 18446744073709551615 134529675 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114526 2654 95 6 0 113192 0
[pid=2262] ppid=2258 vsize=53076 CPUtime=1122.31 cores=3
/proc/2262/stat : 2262 (lingeling) R 2258 2258 379 0 -1 4202496 113706 0 0 0 112215 16 0 0 25 0 1 0 520856586 54349824 12438 33554432000 4194304 4352236 140736224321184 18446744073709551615 4276757 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2262/statm: 13269 12438 127 39 0 12349 0
[pid=2263] ppid=2258 vsize=1410720 CPUtime=558.12 cores=7
/proc/2263/stat : 2263 (march_hi) R 2258 2258 379 0 -1 4202496 785557 0 0 0 55703 109 0 0 25 0 1 0 520856586 1444577280 262186 33554432000 4194304 4331988 140733772606736 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2263/statm: 352680 262186 175 34 0 351122 0
Current children cumulated CPU time (s) 4483.14
Current children cumulated vsize (KiB) 2291044

[startup+1182.3 s]
/proc/loadavg: 5.08 5.08 5.43 6/181 2307
/proc/meminfo: memFree=29474368/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=97028 CPUtime=1182.32 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 85568 0 0 0 118193 39 0 0 25 0 1 0 520856586 99356672 19684 33554432000 4194304 4587354 140733633142512 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 24257 19684 338 96 0 20819 0
[pid=2260] ppid=2258 vsize=164324 CPUtime=1182.32 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 38907 0 0 0 118215 17 0 0 25 0 1 0 520856586 168267776 38212 33554432000 4194304 4785740 140735221116336 18446744073709551615 4550305 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 41081 38212 340 145 0 38126 0
[pid=2261] ppid=2258 vsize=458104 CPUtime=587.95 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 2757 0 0 0 58795 0 0 0 25 0 1 0 520856586 469098496 2654 33554432000 134512640 134534564 4292978880 18446744073709551615 134528913 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114526 2654 95 6 0 113192 0
[pid=2262] ppid=2258 vsize=54104 CPUtime=1182.31 cores=3
/proc/2262/stat : 2262 (lingeling) R 2258 2258 379 0 -1 4202496 121131 0 0 0 118214 17 0 0 25 0 1 0 520856586 55402496 12695 33554432000 4194304 4352236 140736224321184 18446744073709551615 4280960 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2262/statm: 13526 12695 127 39 0 12606 0
[pid=2263] ppid=2258 vsize=1410720 CPUtime=587.92 cores=7
/proc/2263/stat : 2263 (march_hi) R 2258 2258 379 0 -1 4202496 785597 0 0 0 58683 109 0 0 25 0 1 0 520856586 1444577280 262226 33554432000 4194304 4331988 140733772606736 18446744073709551615 4211004 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2263/statm: 352680 262226 175 34 0 351122 0
Current children cumulated CPU time (s) 4722.82
Current children cumulated vsize (KiB) 2202604



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 5.05 5.08 5.42 6/181 2308
/proc/meminfo: memFree=29470132/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=97028 CPUtime=1201.72 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 86668 0 0 0 120133 39 0 0 25 0 1 0 520856586 99356672 20769 33554432000 4194304 4587354 140733633142512 18446744073709551615 4238719 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 24257 20769 338 96 0 20819 0
[pid=2260] ppid=2258 vsize=164324 CPUtime=1201.73 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 38908 0 0 0 120156 17 0 0 25 0 1 0 520856586 168267776 38213 33554432000 4194304 4785740 140735221116336 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 41081 38213 340 145 0 38126 0
[pid=2261] ppid=2258 vsize=458104 CPUtime=597.55 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 2757 0 0 0 59755 0 0 0 25 0 1 0 520856586 469098496 2654 33554432000 134512640 134534564 4292978880 18446744073709551615 134528687 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114526 2654 95 6 0 113192 0
[pid=2262] ppid=2258 vsize=53080 CPUtime=1201.71 cores=3
/proc/2262/stat : 2262 (lingeling) R 2258 2258 379 0 -1 4202496 124715 0 0 0 120154 17 0 0 25 0 1 0 520856586 54353920 12439 33554432000 4194304 4352236 140736224321184 18446744073709551615 4328100 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2262/statm: 13270 12439 127 39 0 12350 0
[pid=2263] ppid=2258 vsize=1410720 CPUtime=597.61 cores=7
/proc/2263/stat : 2263 (march_hi) R 2258 2258 379 0 -1 4202496 785597 0 0 0 59652 109 0 0 25 0 1 0 520856586 1444577280 262226 33554432000 4194304 4331988 140733772606736 18446744073709551615 4210977 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2263/statm: 352680 262226 175 34 0 351122 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 2201580

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 5.05 5.08 5.42 6/181 2308
/proc/meminfo: memFree=29470132/32951124 swapFree=67111344/67111528
[pid=2258] ppid=2256 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2258/stat : 2258 (ppfolio) S 2256 2258 379 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 520856585 18763776 390 33554432000 4194304 4287625 140734358179968 18446744073709551615 271550557711 0 0 4096 65536 4594428637968596991 0 0 17 3 0 0 0
/proc/2258/statm: 4581 390 341 23 0 91 0
[pid=2259] ppid=2258 vsize=97028 CPUtime=1201.72 cores=1
/proc/2259/stat : 2259 (cryptominisat) R 2258 2258 379 0 -1 4202496 86668 0 0 0 120133 39 0 0 25 0 1 0 520856586 99356672 20769 33554432000 4194304 4587354 140733633142512 18446744073709551615 4238719 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2259/statm: 24257 20769 338 96 0 20819 0
[pid=2260] ppid=2258 vsize=164324 CPUtime=1201.73 cores=5
/proc/2260/stat : 2260 (clasp) R 2258 2258 379 0 -1 4202496 38908 0 0 0 120156 17 0 0 25 0 1 0 520856586 168267776 38213 33554432000 4194304 4785740 140735221116336 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2260/statm: 41081 38213 340 145 0 38126 0
[pid=2261] ppid=2258 vsize=458104 CPUtime=597.55 cores=7
/proc/2261/stat : 2261 (TNM) R 2258 2258 379 0 -1 4202496 2757 0 0 0 59755 0 0 0 25 0 1 0 520856586 469098496 2654 33554432000 134512640 134534564 4292978880 18446744073709551615 134528687 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2261/statm: 114526 2654 95 6 0 113192 0
[pid=2262] ppid=2258 vsize=53080 CPUtime=1201.71 cores=3
/proc/2262/stat : 2262 (lingeling) R 2258 2258 379 0 -1 4202496 124715 0 0 0 120154 17 0 0 25 0 1 0 520856586 54353920 12439 33554432000 4194304 4352236 140736224321184 18446744073709551615 4328100 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2262/statm: 13270 12439 127 39 0 12350 0
[pid=2263] ppid=2258 vsize=1410720 CPUtime=597.61 cores=7
/proc/2263/stat : 2263 (march_hi) R 2258 2258 379 0 -1 4202496 785597 0 0 0 59652 109 0 0 25 0 1 0 520856586 1444577280 262226 33554432000 4194304 4331988 140733772606736 18446744073709551615 4210977 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2263/statm: 352680 262226 175 34 0 351122 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 2201580

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 2258 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=3999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=7998
# CPU time returned by wait4() is 0.011997
# while last known CPU time is 4800.32
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.76
CPU time (s): 4800.32
CPU user time (s): 4798.5
CPU system time (s): 1.82
CPU usage (%): 399.442
Max. virtual memory (cumulated for all children) (KiB): 2395432

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.003999
system time used= 0.007998
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 523
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 245
involuntary context switches= 6

runsolver used 1.72274 second user time and 4.84226 second system time

The end

Launcher Data 

Begin job on node133 at 2011-04-29 17:56:27
IDJOB=3291422
IDBENCH=84215
IDSOLVER=1612
FILE ID=node133/3291422-1304092587
RUNJOBID= node133-1304088977-1975
PBS_JOBID= 13170511
Free space on /tmp= 71896 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT03/industrial/goldberg/hard_eq_check/k2mul.miter.shuffled-as.sat03-355.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3291422-1304092587/watcher-3291422-1304092587 -o /tmp/evaluation-result-3291422-1304092587/solver-3291422-1304092587 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3291422-1304092587.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 77e0700cbaeeb3d63a4820608fc84761
RANDOM SEED=1463957766

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30817256 kB
Buffers:        749216 kB
Cached:         964764 kB
SwapCached:        124 kB
Active:         603928 kB
Inactive:      1173464 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30817256 kB
SwapTotal:    67111528 kB
SwapFree:     67111344 kB
Dirty:           11704 kB
Writeback:           0 kB
AnonPages:       62800 kB
Mapped:          14720 kB
Slab:           294052 kB
PageTables:       4156 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   188344 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71884 MiB
End job on node133 at 2011-04-29 18:16:31