Trace number 3190847

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.36 1201.71

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v40000-c168000-S471691933-060-UNKNOWN.cnf
MD5SUM91da78a39fd68b6c63602d278464956f
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables40000
Number of clauses168000
Sum of the clauses size504000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3168000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.00	Starting clasp 1.3.6 on core(s) 5
0.00/0.00	Starting TNM 2009 on core(s) 7
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.00	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3190847-1304038163/watcher-3190847-1304038163 -o /tmp/evaluation-result-3190847-1304038163/solver-3190847-1304038163 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3190847-1304038163.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.19 5.12 5.51 1/171 18027
/proc/meminfo: memFree=12794336/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) R 18025 18027 16932 0 -1 4202496 521 0 0 0 0 0 0 0 21 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 0 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0

[startup+0.26782 s]
/proc/loadavg: 5.19 5.12 5.51 1/171 18027
/proc/meminfo: memFree=12794336/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 21 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.467785 s]
/proc/loadavg: 5.19 5.12 5.51 1/171 18027
/proc/meminfo: memFree=12794336/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 19 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.567764 s]
/proc/loadavg: 5.19 5.12 5.51 1/171 18027
/proc/meminfo: memFree=12794336/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 19 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700742 s]
/proc/loadavg: 5.19 5.12 5.51 1/171 18027
/proc/meminfo: memFree=12794336/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 19 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50059 s]
/proc/loadavg: 5.19 5.12 5.51 6/177 18034
/proc/meminfo: memFree=12667200/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=60336 CPUtime=1.49 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 13689 0 0 0 146 3 0 0 21 0 1 0 594047011 61784064 10502 33554432000 4194304 4587354 140734149755216 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 15084 10502 326 96 0 11646 0
[pid=18029] ppid=18027 vsize=33776 CPUtime=1.49 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 7745 0 0 0 148 1 0 0 23 0 1 0 594047011 34586624 5657 33554432000 4194304 4785740 140734374160112 18446744073709551615 4546313 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 8444 5657 340 145 0 5489 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=0.76 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 76 0 0 0 22 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 134528893 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=19408 CPUtime=1.49 cores=3
/proc/18031/stat : 18031 (lingeling) R 18027 18027 16932 0 -1 4202496 6670 0 0 0 148 1 0 0 20 0 1 0 594047011 19873792 3508 33554432000 4194304 4352236 140735604311216 18446744073709551615 4331567 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/18031/statm: 4852 3508 127 39 0 3932 0
[pid=18032] ppid=18027 vsize=29604 CPUtime=0.7 cores=7
/proc/18032/stat : 18032 (march_hi) R 18027 18027 16932 0 -1 4194304 14889 0 0 0 68 2 0 0 25 0 1 0 594047011 30314496 4198 33554432000 4194304 4331988 140737078493216 18446744073709551615 256415088818 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18032/statm: 7401 4198 129 34 0 5843 0
Current children cumulated CPU time (s) 5.93
Current children cumulated vsize (KiB) 629416

[startup+3.10129 s]
/proc/loadavg: 5.19 5.12 5.51 6/177 18034
/proc/meminfo: memFree=12664236/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=97840 CPUtime=3.09 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 14633 0 0 0 306 3 0 0 25 0 1 0 594047011 100188160 11445 33554432000 4194304 4587354 140734149755216 18446744073709551615 4241083 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 24460 11445 326 96 0 21022 0
[pid=18029] ppid=18027 vsize=35640 CPUtime=3.09 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 8276 0 0 0 308 1 0 0 25 0 1 0 594047011 36495360 6188 33554432000 4194304 4785740 140734374160112 18446744073709551615 4552970 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 8910 6188 340 145 0 5955 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=1.56 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 156 0 0 0 25 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 3288291 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=19408 CPUtime=3.09 cores=3
/proc/18031/stat : 18031 (lingeling) R 18027 18027 16932 0 -1 4202496 6670 0 0 0 308 1 0 0 25 0 1 0 594047011 19873792 3508 33554432000 4194304 4352236 140735604311216 18446744073709551615 4276741 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/18031/statm: 4852 3508 127 39 0 3932 0
[pid=18032] ppid=18027 vsize=31940 CPUtime=1.51 cores=7
/proc/18032/stat : 18032 (march_hi) R 18027 18027 16932 0 -1 4202496 31961 0 0 0 146 5 0 0 25 0 1 0 594047011 32706560 4893 33554432000 4194304 4331988 140737078493216 18446744073709551615 4252086 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18032/statm: 7985 4893 143 34 0 6427 0
Current children cumulated CPU time (s) 12.34
Current children cumulated vsize (KiB) 671120

[startup+6.30068 s]
/proc/loadavg: 5.17 5.12 5.51 6/177 18034
/proc/meminfo: memFree=12604096/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=97840 CPUtime=6.29 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 16274 0 0 0 625 4 0 0 25 0 1 0 594047011 100188160 13081 33554432000 4194304 4587354 140734149755216 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 24460 13081 326 96 0 21022 0
[pid=18029] ppid=18027 vsize=48900 CPUtime=6.28 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 11557 0 0 0 626 2 0 0 25 0 1 0 594047011 50073600 9469 33554432000 4194304 4785740 140734374160112 18446744073709551615 256415052485 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 12225 9469 340 145 0 9270 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=3.14 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 314 0 0 0 25 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 134526045 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=19408 CPUtime=6.29 cores=3
/proc/18031/stat : 18031 (lingeling) R 18027 18027 16932 0 -1 4202496 6670 0 0 0 628 1 0 0 25 0 1 0 594047011 19873792 3508 33554432000 4194304 4352236 140735604311216 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/18031/statm: 4852 3508 127 39 0 3932 0
[pid=18032] ppid=18027 vsize=86588 CPUtime=3.1 cores=7
/proc/18032/stat : 18032 (march_hi) R 18027 18027 16932 0 -1 4202496 55479 0 0 0 302 8 0 0 25 0 1 0 594047011 88666112 18118 33554432000 4194304 4331988 140737078493216 18446744073709551615 4229042 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18032/statm: 21647 18118 173 34 0 20089 0
Current children cumulated CPU time (s) 25.1
Current children cumulated vsize (KiB) 739028

[startup+12.7005 s]
/proc/loadavg: 5.16 5.12 5.51 6/177 18034
/proc/meminfo: memFree=12573376/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=69464 CPUtime=12.69 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 21372 0 0 0 1263 6 0 0 25 0 1 0 594047011 71131136 11311 33554432000 4194304 4587354 140734149755216 18446744073709551615 4243380 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 17366 11311 327 96 0 13928 0
[pid=18029] ppid=18027 vsize=69268 CPUtime=12.69 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 16668 0 0 0 1265 4 0 0 25 0 1 0 594047011 70930432 14580 33554432000 4194304 4785740 140734374160112 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 17317 14580 340 145 0 14362 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=6.34 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 634 0 0 0 25 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 134528412 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=24672 CPUtime=12.69 cores=3
/proc/18031/stat : 18031 (lingeling) R 18027 18027 16932 0 -1 4202496 8497 0 0 0 1268 1 0 0 25 0 1 0 594047011 25264128 5335 33554432000 4194304 4352236 140735604311216 18446744073709551615 4329086 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/18031/statm: 6168 5335 127 39 0 5248 0
[pid=18032] ppid=18027 vsize=88144 CPUtime=6.27 cores=7
/proc/18032/stat : 18032 (march_hi) R 18027 18027 16932 0 -1 4202496 55852 0 0 0 619 8 0 0 25 0 1 0 594047011 90259456 18491 33554432000 4194304 4331988 140737078493216 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18032/statm: 22036 18491 173 34 0 20478 0
Current children cumulated CPU time (s) 50.68
Current children cumulated vsize (KiB) 737840

[startup+25.5011 s]
/proc/loadavg: 5.12 5.11 5.50 6/177 18034
/proc/meminfo: memFree=12506108/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=143468 CPUtime=25.49 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 49462 0 0 0 2537 12 0 0 25 0 1 0 594047011 146911232 15856 33554432000 4194304 4587354 140734149755216 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 35867 15856 337 96 0 32429 0
[pid=18029] ppid=18027 vsize=108156 CPUtime=25.49 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 26325 0 0 0 2541 8 0 0 25 0 1 0 594047011 110751744 24237 33554432000 4194304 4785740 140734374160112 18446744073709551615 256415052485 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 27039 24237 340 145 0 24084 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=12.74 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 1274 0 0 0 25 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=24672 CPUtime=25.49 cores=3
/proc/18031/stat : 18031 (lingeling) R 18027 18027 16932 0 -1 4202496 8497 0 0 0 2548 1 0 0 25 0 1 0 594047011 25264128 5335 33554432000 4194304 4352236 140735604311216 18446744073709551615 4328100 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/18031/statm: 6168 5335 127 39 0 5248 0
[pid=18032] ppid=18027 vsize=98720 CPUtime=12.61 cores=7
/proc/18032/stat : 18032 (march_hi) R 18027 18027 16932 0 -1 4202496 58066 0 0 0 1252 9 0 0 25 0 1 0 594047011 101089280 20705 33554432000 4194304 4331988 140737078493216 18446744073709551615 4288411 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18032/statm: 24680 20705 173 34 0 23122 0
Current children cumulated CPU time (s) 101.82
Current children cumulated vsize (KiB) 861308

[startup+51.1012 s]
/proc/loadavg: 5.08 5.10 5.49 6/177 18035
/proc/meminfo: memFree=12393736/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=142100 CPUtime=51.09 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 92372 0 0 0 5085 24 0 0 25 0 1 0 594047011 145510400 30606 33554432000 4194304 4587354 140734149755216 18446744073709551615 4352114 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 35525 30606 337 96 0 32087 0
[pid=18029] ppid=18027 vsize=156084 CPUtime=51.09 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 38188 0 0 0 5098 11 0 0 25 0 1 0 594047011 159830016 36100 33554432000 4194304 4785740 140734374160112 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 39021 36100 340 145 0 36066 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=25.43 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 2543 0 0 0 25 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 134529640 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=29772 CPUtime=51.09 cores=3

################
# More data... #
################

/proc/18032/statm: 26264 22289 173 34 0 24706 0
Current children cumulated CPU time (s) 3524.46
Current children cumulated vsize (KiB) 1467004

[startup+942.301 s]
/proc/loadavg: 5.08 5.07 5.21 6/177 18068
/proc/meminfo: memFree=12127368/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=1047932 CPUtime=942.32 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 610675 0 0 0 93992 240 0 0 25 0 1 0 594047011 1073082368 18129 33554432000 4194304 4587354 140734149755216 18446744073709551615 4238101 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 261983 18129 337 96 0 258545 0
[pid=18029] ppid=18027 vsize=469052 CPUtime=942.31 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 114980 0 0 0 94191 40 0 0 25 0 1 0 594047011 480309248 112892 33554432000 4194304 4785740 140734374160112 18446744073709551615 4278573 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 117263 112892 340 145 0 114308 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=468.64 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 46864 0 0 0 25 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 134530504 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=33036 CPUtime=942.32 cores=3
/proc/18031/stat : 18031 (lingeling) R 18027 18027 16932 0 -1 4202496 10588 0 0 0 94231 1 0 0 25 0 1 0 594047011 33828864 7426 33554432000 4194304 4352236 140735604311216 18446744073709551615 4276757 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/18031/statm: 8259 7426 127 39 0 7339 0
[pid=18032] ppid=18027 vsize=105056 CPUtime=468.53 cores=7
/proc/18032/stat : 18032 (march_hi) R 18027 18027 16932 0 -1 4202496 59679 0 0 0 46843 10 0 0 25 0 1 0 594047011 107577344 22318 33554432000 4194304 4331988 140737078493216 18446744073709551615 4226496 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18032/statm: 26264 22318 173 34 0 24706 0
Current children cumulated CPU time (s) 3764.12
Current children cumulated vsize (KiB) 2141368

[startup+1002.3 s]
/proc/loadavg: 5.06 5.07 5.20 6/177 18069
/proc/meminfo: memFree=12089884/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=1022720 CPUtime=1002.33 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 631830 0 0 0 99980 253 0 0 25 0 1 0 594047011 1047265280 27250 33554432000 4194304 4587354 140734149755216 18446744073709551615 256415088284 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 255680 27250 337 96 0 252242 0
[pid=18029] ppid=18027 vsize=469052 CPUtime=1002.32 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 114993 0 0 0 100192 40 0 0 25 0 1 0 594047011 480309248 112905 33554432000 4194304 4785740 140734374160112 18446744073709551615 4309555 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 117263 112905 340 145 0 114308 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=498.46 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 49846 0 0 0 25 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 134528913 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=33036 CPUtime=1002.33 cores=3
/proc/18031/stat : 18031 (lingeling) R 18027 18027 16932 0 -1 4202496 10588 0 0 0 100232 1 0 0 25 0 1 0 594047011 33828864 7426 33554432000 4194304 4352236 140735604311216 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/18031/statm: 8259 7426 127 39 0 7339 0
[pid=18032] ppid=18027 vsize=105188 CPUtime=498.39 cores=7
/proc/18032/stat : 18032 (march_hi) R 18027 18027 16932 0 -1 4202496 59703 0 0 0 49829 10 0 0 25 0 1 0 594047011 107712512 22342 33554432000 4194304 4331988 140737078493216 18446744073709551615 4240754 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18032/statm: 26297 22342 173 34 0 24739 0
Current children cumulated CPU time (s) 4003.83
Current children cumulated vsize (KiB) 2116288

[startup+1062.3 s]
/proc/loadavg: 5.11 5.08 5.19 6/177 18071
/proc/meminfo: memFree=12083660/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=1055988 CPUtime=1062.32 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 654416 0 0 0 105964 268 0 0 25 0 1 0 594047011 1081331712 24987 33554432000 4194304 4587354 140734149755216 18446744073709551615 4238323 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 263997 24987 337 96 0 260559 0
[pid=18029] ppid=18027 vsize=484784 CPUtime=1062.32 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 118564 0 0 0 106192 40 0 0 25 0 1 0 594047011 496418816 116476 33554432000 4194304 4785740 140734374160112 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 121196 116476 340 145 0 118241 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=528.3 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 52830 0 0 0 25 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 134526128 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=35084 CPUtime=1062.32 cores=3
/proc/18031/stat : 18031 (lingeling) R 18027 18027 16932 0 -1 4202496 11100 0 0 0 106231 1 0 0 25 0 1 0 594047011 35926016 7938 33554432000 4194304 4352236 140735604311216 18446744073709551615 4231080 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/18031/statm: 8771 7938 127 39 0 7851 0
[pid=18032] ppid=18027 vsize=105320 CPUtime=528.22 cores=7
/proc/18032/stat : 18032 (march_hi) R 18027 18027 16932 0 -1 4202496 59728 0 0 0 52812 10 0 0 25 0 1 0 594047011 107847680 22367 33554432000 4194304 4331988 140737078493216 18446744073709551615 4292195 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18032/statm: 26330 22367 173 34 0 24772 0
Current children cumulated CPU time (s) 4243.48
Current children cumulated vsize (KiB) 2167468

[startup+1122.31 s]
/proc/loadavg: 5.03 5.06 5.18 6/177 18072
/proc/meminfo: memFree=12029196/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=1040256 CPUtime=1122.34 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 675172 0 0 0 111951 283 0 0 25 0 1 0 594047011 1065222144 32795 33554432000 4194304 4587354 140734149755216 18446744073709551615 4236912 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 260064 32795 337 96 0 256626 0
[pid=18029] ppid=18027 vsize=506928 CPUtime=1122.32 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 124104 0 0 0 112191 41 0 0 25 0 1 0 594047011 519094272 122016 33554432000 4194304 4785740 140734374160112 18446744073709551615 4553068 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 126732 122016 340 145 0 123777 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=558.15 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 55815 0 0 0 25 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 134528913 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=35084 CPUtime=1122.33 cores=3
/proc/18031/stat : 18031 (lingeling) R 18027 18027 16932 0 -1 4202496 11100 0 0 0 112232 1 0 0 25 0 1 0 594047011 35926016 7938 33554432000 4194304 4352236 140735604311216 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/18031/statm: 8771 7938 127 39 0 7851 0
[pid=18032] ppid=18027 vsize=105452 CPUtime=558.04 cores=7
/proc/18032/stat : 18032 (march_hi) R 18027 18027 16932 0 -1 4202496 59754 0 0 0 55794 10 0 0 25 0 1 0 594047011 107982848 22393 33554432000 4194304 4331988 140737078493216 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18032/statm: 26363 22393 173 34 0 24805 0
Current children cumulated CPU time (s) 4483.18
Current children cumulated vsize (KiB) 2174012

[startup+1182.3 s]
/proc/loadavg: 5.05 5.06 5.17 6/177 18074
/proc/meminfo: memFree=12057556/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=1055820 CPUtime=1182.33 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 693969 0 0 0 117936 297 0 0 25 0 1 0 594047011 1081159680 24885 33554432000 4194304 4587354 140734149755216 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 263955 24885 337 96 0 260517 0
[pid=18029] ppid=18027 vsize=509248 CPUtime=1182.32 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 124705 0 0 0 118190 42 0 0 25 0 1 0 594047011 521469952 122617 33554432000 4194304 4785740 140734374160112 18446744073709551615 4287201 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 127312 122617 340 145 0 124357 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=587.96 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 58796 0 0 0 25 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 134528504 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=35084 CPUtime=1182.33 cores=3
/proc/18031/stat : 18031 (lingeling) R 18027 18027 16932 0 -1 4202496 11100 0 0 0 118232 1 0 0 25 0 1 0 594047011 35926016 7938 33554432000 4194304 4352236 140735604311216 18446744073709551615 4328047 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/18031/statm: 8771 7938 127 39 0 7851 0
[pid=18032] ppid=18027 vsize=105452 CPUtime=587.91 cores=7
/proc/18032/stat : 18032 (march_hi) R 18027 18027 16932 0 -1 4202496 59775 0 0 0 58781 10 0 0 25 0 1 0 594047011 107982848 22414 33554432000 4194304 4331988 140737078493216 18446744073709551615 4226474 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18032/statm: 26363 22414 173 34 0 24805 0
Current children cumulated CPU time (s) 4722.85
Current children cumulated vsize (KiB) 2191896



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 5.03 5.06 5.17 6/177 18075
/proc/meminfo: memFree=12032384/32950928 swapFree=67111524/67111528
[pid=18027] ppid=18025 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/18027/stat : 18027 (ppfolio) S 18025 18027 16932 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 594047011 18763776 389 33554432000 4194304 4287625 140733425153056 18446744073709551615 256415411727 0 0 4096 65536 4348613297247879167 0 0 17 5 0 0 0
/proc/18027/statm: 4581 389 341 23 0 91 0
[pid=18028] ppid=18027 vsize=1055820 CPUtime=1201.74 cores=1
/proc/18028/stat : 18028 (cryptominisat) R 18027 18027 16932 0 -1 4202496 700415 0 0 0 119872 302 0 0 25 0 1 0 594047011 1081159680 31304 33554432000 4194304 4587354 140734149755216 18446744073709551615 4241080 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/18028/statm: 263955 31304 337 96 0 260517 0
[pid=18029] ppid=18027 vsize=509248 CPUtime=1201.73 cores=5
/proc/18029/stat : 18029 (clasp) R 18027 18027 16932 0 -1 4202496 124706 0 0 0 120131 42 0 0 25 0 1 0 594047011 521469952 122618 33554432000 4194304 4785740 140734374160112 18446744073709551615 4553068 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/18029/statm: 127312 122618 340 145 0 124357 0
[pid=18030] ppid=18027 vsize=467968 CPUtime=597.65 cores=7
/proc/18030/stat : 18030 (TNM) R 18027 18027 16932 0 -1 4202496 6720 0 0 0 59765 0 0 0 25 0 1 0 594047011 479199232 6616 33554432000 134512640 134534564 4290908528 18446744073709551615 134528979 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18030/statm: 116992 6616 95 6 0 115658 0
[pid=18031] ppid=18027 vsize=35084 CPUtime=1201.73 cores=3
/proc/18031/stat : 18031 (lingeling) R 18027 18027 16932 0 -1 4202496 11100 0 0 0 120172 1 0 0 25 0 1 0 594047011 35926016 7938 33554432000 4194304 4352236 140735604311216 18446744073709551615 4277685 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/18031/statm: 8771 7938 127 39 0 7851 0
[pid=18032] ppid=18027 vsize=105584 CPUtime=597.51 cores=7
/proc/18032/stat : 18032 (march_hi) R 18027 18027 16932 0 -1 4202496 59782 0 0 0 59741 10 0 0 25 0 1 0 594047011 108118016 22421 33554432000 4194304 4331988 140737078493216 18446744073709551615 4211509 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/18032/statm: 26396 22421 173 34 0 24838 0
Current children cumulated CPU time (s) 4800.36
Current children cumulated vsize (KiB) 2192028

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 18027 and gives
#  childrusage.ru_utime.tv_sec=1201
#  childrusage.ru_utime.tv_usec=732308
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=29995
# CPU time returned by wait4() is 1201.76
# while last known CPU time is 4800.36
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.36
CPU user time (s): 4796.81
CPU system time (s): 3.55
CPU usage (%): 399.46
Max. virtual memory (cumulated for all children) (KiB): 3116160

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1201.73
system time used= 0.029995
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 11624
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 236
involuntary context switches= 1805

runsolver used 1.75173 second user time and 4.81327 second system time

The end

Launcher Data 

Begin job on node115 at 2011-04-29 02:49:23
IDJOB=3190847
IDBENCH=82578
IDSOLVER=1612
FILE ID=node115/3190847-1304038163
RUNJOBID= node115-1304032879-17473
PBS_JOBID= 13170337
Free space on /tmp= 72728 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v40000-c168000-S471691933-060-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3190847-1304038163/watcher-3190847-1304038163 -o /tmp/evaluation-result-3190847-1304038163/solver-3190847-1304038163 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3190847-1304038163.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 91da78a39fd68b6c63602d278464956f
RANDOM SEED=646234471

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      12794492 kB
Buffers:       2315204 kB
Cached:       17220912 kB
SwapCached:          4 kB
Active:        7587776 kB
Inactive:     12012104 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      12794492 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           13548 kB
Writeback:          20 kB
AnonPages:       63372 kB
Mapped:          14500 kB
Slab:           492564 kB
PageTables:       4184 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   184060 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72720 MiB
End job on node115 at 2011-04-29 03:09:27