Trace number 3191037

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.35 1201.71

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v600-c2556-S1245124682-071.UNKNOWN.cnf
MD5SUMb386c0165fb504493e3604d6c778ca42
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables600
Number of clauses2556
Sum of the clauses size7668
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32556
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.00	Starting clasp 1.3.6 on core(s) 5
0.00/0.00	Starting TNM 2009 on core(s) 7
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.00	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3191037-1304053979/watcher-3191037-1304053979 -o /tmp/evaluation-result-3191037-1304053979/solver-3191037-1304053979 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3191037-1304053979.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.04 5.07 5.15 1/175 29567
/proc/meminfo: memFree=31054628/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 23 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0

[startup+1.06805 s]
/proc/loadavg: 5.04 5.07 5.15 1/175 29567
/proc/meminfo: memFree=31054628/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) R 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 21 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 0 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.16803 s]
/proc/loadavg: 5.04 5.07 5.15 1/175 29567
/proc/meminfo: memFree=31054628/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) R 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 21 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 0 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.20103 s]
/proc/loadavg: 5.04 5.07 5.15 1/175 29567
/proc/meminfo: memFree=31054628/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) R 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 21 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 0 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.36799 s]
/proc/loadavg: 5.04 5.07 5.15 1/175 29567
/proc/meminfo: memFree=31054628/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 20 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.56796 s]
/proc/loadavg: 5.04 5.07 5.15 1/175 29567
/proc/meminfo: memFree=31054628/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 18 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+3.10068 s]
/proc/loadavg: 5.12 5.09 5.15 6/181 29573
/proc/meminfo: memFree=31032528/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=1760932 CPUtime=3.09 cores=1
/proc/29568/stat : 29568 (cryptominisat) R 29567 29567 26347 0 -1 4202496 5525 0 0 0 309 0 0 0 25 0 1 0 516995784 1803194368 3299 33554432000 4194304 4587354 140736659967248 18446744073709551615 4357604 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/29568/statm: 440233 3299 337 96 0 436795 0
[pid=29569] ppid=29567 vsize=14972 CPUtime=3.09 cores=5
/proc/29569/stat : 29569 (clasp) R 29567 29567 26347 0 -1 4202496 1212 0 0 0 309 0 0 0 25 0 1 0 516995784 15331328 1099 33554432000 4194304 4785740 140736907391888 18446744073709551615 4287092 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/29569/statm: 3743 1099 340 145 0 788 0
[pid=29570] ppid=29567 vsize=455960 CPUtime=1.97 cores=7
/proc/29570/stat : 29570 (TNM) R 29567 29567 26347 0 -1 4202496 355 0 0 0 197 0 0 0 21 0 1 0 516995784 466903040 252 33554432000 134512640 134534564 4290972672 18446744073709551615 134529080 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29570/statm: 113990 252 95 6 0 112656 0
[pid=29571] ppid=29567 vsize=6588 CPUtime=3.09 cores=3
/proc/29571/stat : 29571 (lingeling) R 29567 29567 26347 0 -1 4202496 1523 0 0 0 309 0 0 0 25 0 1 0 516995784 6746112 844 33554432000 4194304 4352236 140735996121328 18446744073709551615 4328385 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/29571/statm: 1647 844 127 39 0 727 0
[pid=29572] ppid=29567 vsize=8156 CPUtime=1.11 cores=7
/proc/29572/stat : 29572 (march_hi) R 29567 29567 26347 0 -1 4202496 662 0 0 0 111 0 0 0 25 0 1 0 516995784 8351744 555 33554432000 4194304 4331988 140736405780256 18446744073709551615 4229623 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29572/statm: 2039 555 173 34 0 481 0
Current children cumulated CPU time (s) 12.35
Current children cumulated vsize (KiB) 2264932

[startup+6.3011 s]
/proc/loadavg: 5.12 5.09 5.15 6/181 29573
/proc/meminfo: memFree=31028808/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=1762868 CPUtime=6.29 cores=1
/proc/29568/stat : 29568 (cryptominisat) R 29567 29567 26347 0 -1 4202496 7701 0 0 0 628 1 0 0 25 0 1 0 516995784 1805176832 3764 33554432000 4194304 4587354 140736659967248 18446744073709551615 4239320 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/29568/statm: 440717 3764 337 96 0 437279 0
[pid=29569] ppid=29567 vsize=16188 CPUtime=6.29 cores=5
/proc/29569/stat : 29569 (clasp) R 29567 29567 26347 0 -1 4202496 1506 0 0 0 629 0 0 0 25 0 1 0 516995784 16576512 1393 33554432000 4194304 4785740 140736907391888 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/29569/statm: 4047 1393 340 145 0 1092 0
[pid=29570] ppid=29567 vsize=455960 CPUtime=3.55 cores=7
/proc/29570/stat : 29570 (TNM) R 29567 29567 26347 0 -1 4202496 355 0 0 0 355 0 0 0 25 0 1 0 516995784 466903040 252 33554432000 134512640 134534564 4290972672 18446744073709551615 134530271 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29570/statm: 113990 252 95 6 0 112656 0
[pid=29571] ppid=29567 vsize=7100 CPUtime=6.29 cores=3
/proc/29571/stat : 29571 (lingeling) R 29567 29567 26347 0 -1 4202496 2195 0 0 0 629 0 0 0 25 0 1 0 516995784 7270400 972 33554432000 4194304 4352236 140735996121328 18446744073709551615 4328213 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/29571/statm: 1775 972 127 39 0 855 0
[pid=29572] ppid=29567 vsize=8408 CPUtime=2.71 cores=7
/proc/29572/stat : 29572 (march_hi) R 29567 29567 26347 0 -1 4202496 720 0 0 0 271 0 0 0 25 0 1 0 516995784 8609792 613 33554432000 4194304 4331988 140736405780256 18446744073709551615 4229310 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29572/statm: 2102 613 173 34 0 544 0
Current children cumulated CPU time (s) 25.13
Current children cumulated vsize (KiB) 2268848

[startup+12.7009 s]
/proc/loadavg: 5.10 5.08 5.15 6/181 29573
/proc/meminfo: memFree=31022500/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=1765864 CPUtime=12.68 cores=1
/proc/29568/stat : 29568 (cryptominisat) R 29567 29567 26347 0 -1 4202496 12037 0 0 0 1267 1 0 0 25 0 1 0 516995784 1808244736 4257 33554432000 4194304 4587354 140736659967248 18446744073709551615 4238571 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/29568/statm: 441466 4257 337 96 0 438028 0
[pid=29569] ppid=29567 vsize=17968 CPUtime=12.69 cores=5
/proc/29569/stat : 29569 (clasp) R 29567 29567 26347 0 -1 4202496 1930 0 0 0 1269 0 0 0 25 0 1 0 516995784 18399232 1817 33554432000 4194304 4785740 140736907391888 18446744073709551615 4553127 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/29569/statm: 4492 1817 341 145 0 1537 0
[pid=29570] ppid=29567 vsize=455960 CPUtime=6.74 cores=7
/proc/29570/stat : 29570 (TNM) R 29567 29567 26347 0 -1 4202496 355 0 0 0 674 0 0 0 25 0 1 0 516995784 466903040 252 33554432000 134512640 134534564 4290972672 18446744073709551615 134528918 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29570/statm: 113990 252 95 6 0 112656 0
[pid=29571] ppid=29567 vsize=8772 CPUtime=12.69 cores=3
/proc/29571/stat : 29571 (lingeling) R 29567 29567 26347 0 -1 4202496 3413 0 0 0 1269 0 0 0 25 0 1 0 516995784 8982528 1390 33554432000 4194304 4352236 140735996121328 18446744073709551615 4328000 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/29571/statm: 2193 1390 127 39 0 1273 0
[pid=29572] ppid=29567 vsize=9248 CPUtime=5.88 cores=7
/proc/29572/stat : 29572 (march_hi) R 29567 29567 26347 0 -1 4202496 861 0 0 0 588 0 0 0 25 0 1 0 516995784 9469952 754 33554432000 4194304 4331988 140736405780256 18446744073709551615 4229623 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29572/statm: 2312 754 173 34 0 754 0
Current children cumulated CPU time (s) 50.68
Current children cumulated vsize (KiB) 2276136

[startup+25.5006 s]
/proc/loadavg: 5.16 5.10 5.16 6/181 29574
/proc/meminfo: memFree=31014216/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=1779472 CPUtime=25.48 cores=1
/proc/29568/stat : 29568 (cryptominisat) R 29567 29567 26347 0 -1 4202496 18705 0 0 0 2546 2 0 0 25 0 1 0 516995784 1822179328 5912 33554432000 4194304 4587354 140736659967248 18446744073709551615 4238406 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/29568/statm: 444868 5912 337 96 0 441430 0
[pid=29569] ppid=29567 vsize=19532 CPUtime=25.49 cores=5
/proc/29569/stat : 29569 (clasp) R 29567 29567 26347 0 -1 4202496 2327 0 0 0 2549 0 0 0 25 0 1 0 516995784 20000768 2214 33554432000 4194304 4785740 140736907391888 18446744073709551615 4553029 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/29569/statm: 4883 2214 341 145 0 1928 0
[pid=29570] ppid=29567 vsize=455960 CPUtime=13.14 cores=7
/proc/29570/stat : 29570 (TNM) R 29567 29567 26347 0 -1 4202496 355 0 0 0 1314 0 0 0 25 0 1 0 516995784 466903040 252 33554432000 134512640 134534564 4290972672 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29570/statm: 113990 252 95 6 0 112656 0
[pid=29571] ppid=29567 vsize=9932 CPUtime=25.48 cores=3
/proc/29571/stat : 29571 (lingeling) R 29567 29567 26347 0 -1 4202496 6103 0 0 0 2548 0 0 0 25 0 1 0 516995784 10170368 1680 33554432000 4194304 4352236 140735996121328 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/29571/statm: 2483 1680 127 39 0 1563 0
[pid=29572] ppid=29567 vsize=10660 CPUtime=12.21 cores=7
/proc/29572/stat : 29572 (march_hi) R 29567 29567 26347 0 -1 4202496 1099 0 0 0 1221 0 0 0 25 0 1 0 516995784 10915840 992 33554432000 4194304 4331988 140736405780256 18446744073709551615 4286778 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29572/statm: 2665 992 173 34 0 1107 0
Current children cumulated CPU time (s) 101.8
Current children cumulated vsize (KiB) 2293880

[startup+51.101 s]
/proc/loadavg: 5.16 5.10 5.16 6/181 29575
/proc/meminfo: memFree=30994400/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=1800680 CPUtime=51.09 cores=1
/proc/29568/stat : 29568 (cryptominisat) R 29567 29567 26347 0 -1 4202496 23761 0 0 0 5105 4 0 0 25 0 1 0 516995784 1843896320 8664 33554432000 4194304 4587354 140736659967248 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/29568/statm: 450170 8664 337 96 0 446732 0
[pid=29569] ppid=29567 vsize=22036 CPUtime=51.09 cores=5
/proc/29569/stat : 29569 (clasp) R 29567 29567 26347 0 -1 4202496 2926 0 0 0 5109 0 0 0 25 0 1 0 516995784 22564864 2813 33554432000 4194304 4785740 140736907391888 18446744073709551615 4287092 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/29569/statm: 5509 2813 341 145 0 2554 0
[pid=29570] ppid=29567 vsize=455960 CPUtime=25.84 cores=7
/proc/29570/stat : 29570 (TNM) R 29567 29567 26347 0 -1 4202496 355 0 0 0 2584 0 0 0 25 0 1 0 516995784 466903040 252 33554432000 134512640 134534564 4290972672 18446744073709551615 134529640 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29570/statm: 113990 252 95 6 0 112656 0
[pid=29571] ppid=29567 vsize=12112 CPUtime=51.09 cores=3
/proc/29571/stat : 29571 (lingeling) R 29567 29567 26347 0 -1 4202496 10744 0 0 0 5108 1 0 0 25 0 1 0 516995784 12402688 2225 33554432000 4194304 4352236 140735996121328 18446744073709551615 4328009 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/29571/statm: 3028 2225 127 39 0 2108 0
[pid=29572] ppid=29567 vsize=12640 CPUtime=24.97 cores=7
/proc/29572/stat : 29572 (march_hi) R 29567 29567 26347 0 -1 4202496 1575 0 0 0 2497 0 0 0 25 0 1 0 516995784 12943360 1468 33554432000 4194304 4331988 140736405780256 18446744073709551615 4265495 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29572/statm: 3160 1468 173 34 0 1602 0
Current children cumulated CPU time (s) 204.08
Current children cumulated vsize (KiB) 2321752

[startup+102.301 s]
/proc/loadavg: 5.06 5.08 5.15 6/181 29576
/proc/meminfo: memFree=30980240/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=61628 CPUtime=102.28 cores=1

################
# More data... #
################

/proc/29572/statm: 21243 15785 173 34 0 19685 0
Current children cumulated CPU time (s) 3764.1
Current children cumulated vsize (KiB) 2483164

[startup+1002.3 s]
/proc/loadavg: 5.14 5.12 5.09 6/181 29600
/proc/meminfo: memFree=30839240/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=1850012 CPUtime=1002.31 cores=1
/proc/29568/stat : 29568 (cryptominisat) R 29567 29567 26347 0 -1 4202496 101960 0 0 0 100195 36 0 0 25 0 1 0 516995784 1894412288 24440 33554432000 4194304 4587354 140736659967248 18446744073709551615 4238406 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/29568/statm: 462503 24440 337 96 0 459065 0
[pid=29569] ppid=29567 vsize=43784 CPUtime=1002.31 cores=5
/proc/29569/stat : 29569 (clasp) R 29567 29567 26347 0 -1 4202496 7944 0 0 0 100230 1 0 0 25 0 1 0 516995784 44834816 7831 33554432000 4194304 4785740 140736907391888 18446744073709551615 4287153 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/29569/statm: 10946 7831 341 145 0 7991 0
[pid=29570] ppid=29567 vsize=455960 CPUtime=498.92 cores=7
/proc/29570/stat : 29570 (TNM) R 29567 29567 26347 0 -1 4202496 355 0 0 0 49892 0 0 0 25 0 1 0 516995784 466903040 252 33554432000 134512640 134534564 4290972672 18446744073709551615 7638036 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29570/statm: 113990 252 95 6 0 112656 0
[pid=29571] ppid=29567 vsize=24916 CPUtime=1002.31 cores=3
/proc/29571/stat : 29571 (lingeling) R 29567 29567 26347 0 -1 4202496 86489 0 0 0 100219 12 0 0 25 0 1 0 516995784 25513984 5426 33554432000 4194304 4352236 140735996121328 18446744073709551615 4328105 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/29571/statm: 6229 5426 127 39 0 5309 0
[pid=29572] ppid=29567 vsize=84972 CPUtime=497.92 cores=7
/proc/29572/stat : 29572 (march_hi) R 29567 29567 26347 0 -1 4202496 16853 0 0 0 49781 11 0 0 25 0 1 0 516995784 87011328 16746 33554432000 4194304 4331988 140736405780256 18446744073709551615 4211445 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29572/statm: 21243 16746 173 34 0 19685 0
Current children cumulated CPU time (s) 4003.77
Current children cumulated vsize (KiB) 2477968

[startup+1062.3 s]
/proc/loadavg: 5.10 5.11 5.09 6/181 29601
/proc/meminfo: memFree=30826304/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=1840948 CPUtime=1062.32 cores=1
/proc/29568/stat : 29568 (cryptominisat) R 29567 29567 26347 0 -1 4202496 102176 0 0 0 106195 37 0 0 25 0 1 0 516995784 1885130752 24441 33554432000 4194304 4587354 140736659967248 18446744073709551615 4238463 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/29568/statm: 460237 24441 337 96 0 456799 0
[pid=29569] ppid=29567 vsize=44736 CPUtime=1062.32 cores=5
/proc/29569/stat : 29569 (clasp) R 29567 29567 26347 0 -1 4202496 8141 0 0 0 106230 2 0 0 25 0 1 0 516995784 45809664 8028 33554432000 4194304 4785740 140736907391888 18446744073709551615 4552952 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/29569/statm: 11184 8028 341 145 0 8229 0
[pid=29570] ppid=29567 vsize=455960 CPUtime=528.76 cores=7
/proc/29570/stat : 29570 (TNM) R 29567 29567 26347 0 -1 4202496 355 0 0 0 52876 0 0 0 25 0 1 0 516995784 466903040 252 33554432000 134512640 134534564 4290972672 18446744073709551615 134530186 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29570/statm: 113990 252 95 6 0 112656 0
[pid=29571] ppid=29567 vsize=31316 CPUtime=1062.32 cores=3
/proc/29571/stat : 29571 (lingeling) R 29567 29567 26347 0 -1 4202496 90777 0 0 0 106219 13 0 0 25 0 1 0 516995784 32067584 7026 33554432000 4194304 4352236 140735996121328 18446744073709551615 4328363 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/29571/statm: 7829 7026 127 39 0 6909 0
[pid=29572] ppid=29567 vsize=84972 CPUtime=527.76 cores=7
/proc/29572/stat : 29572 (march_hi) R 29567 29567 26347 0 -1 4202496 17752 0 0 0 52764 12 0 0 25 0 1 0 516995784 87011328 17645 33554432000 4194304 4331988 140736405780256 18446744073709551615 4299872 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29572/statm: 21243 17645 173 34 0 19685 0
Current children cumulated CPU time (s) 4243.48
Current children cumulated vsize (KiB) 2476256

[startup+1122.3 s]
/proc/loadavg: 5.12 5.11 5.09 6/181 29603
/proc/meminfo: memFree=30827640/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=1897320 CPUtime=1122.31 cores=1
/proc/29568/stat : 29568 (cryptominisat) R 29567 29567 26347 0 -1 4202496 102753 0 0 0 112194 37 0 0 25 0 1 0 516995784 1942855680 24466 33554432000 4194304 4587354 140736659967248 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/29568/statm: 474330 24466 337 96 0 470892 0
[pid=29569] ppid=29567 vsize=45384 CPUtime=1122.32 cores=5
/proc/29569/stat : 29569 (clasp) R 29567 29567 26347 0 -1 4202496 8298 0 0 0 112230 2 0 0 25 0 1 0 516995784 46473216 8185 33554432000 4194304 4785740 140736907391888 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/29569/statm: 11346 8185 341 145 0 8391 0
[pid=29570] ppid=29567 vsize=455960 CPUtime=558.58 cores=7
/proc/29570/stat : 29570 (TNM) R 29567 29567 26347 0 -1 4202496 355 0 0 0 55858 0 0 0 25 0 1 0 516995784 466903040 252 33554432000 134512640 134534564 4290972672 18446744073709551615 134529031 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29570/statm: 113990 252 95 6 0 112656 0
[pid=29571] ppid=29567 vsize=25940 CPUtime=1122.32 cores=3
/proc/29571/stat : 29571 (lingeling) R 29567 29567 26347 0 -1 4202496 93081 0 0 0 112219 13 0 0 25 0 1 0 516995784 26562560 5682 33554432000 4194304 4352236 140735996121328 18446744073709551615 4329081 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/29571/statm: 6485 5682 127 39 0 5565 0
[pid=29572] ppid=29567 vsize=84972 CPUtime=557.62 cores=7
/proc/29572/stat : 29572 (march_hi) R 29567 29567 26347 0 -1 4202496 18651 0 0 0 55749 13 0 0 25 0 1 0 516995784 87011328 18544 33554432000 4194304 4331988 140736405780256 18446744073709551615 4226434 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29572/statm: 21243 18544 173 34 0 19685 0
Current children cumulated CPU time (s) 4483.15
Current children cumulated vsize (KiB) 2527900

[startup+1182.3 s]
/proc/loadavg: 5.13 5.12 5.09 6/181 29605
/proc/meminfo: memFree=30817568/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=1850012 CPUtime=1182.32 cores=1
/proc/29568/stat : 29568 (cryptominisat) R 29567 29567 26347 0 -1 4202496 103041 0 0 0 118195 37 0 0 25 0 1 0 516995784 1894412288 24481 33554432000 4194304 4587354 140736659967248 18446744073709551615 4238618 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/29568/statm: 462503 24481 337 96 0 459065 0
[pid=29569] ppid=29567 vsize=45892 CPUtime=1182.33 cores=5
/proc/29569/stat : 29569 (clasp) R 29567 29567 26347 0 -1 4202496 8416 0 0 0 118231 2 0 0 25 0 1 0 516995784 46993408 8303 33554432000 4194304 4785740 140736907391888 18446744073709551615 4287158 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/29569/statm: 11473 8303 341 145 0 8518 0
[pid=29570] ppid=29567 vsize=455960 CPUtime=588.46 cores=7
/proc/29570/stat : 29570 (TNM) R 29567 29567 26347 0 -1 4202496 355 0 0 0 58846 0 0 0 25 0 1 0 516995784 466903040 252 33554432000 134512640 134534564 4290972672 18446744073709551615 134528902 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29570/statm: 113990 252 95 6 0 112656 0
[pid=29571] ppid=29567 vsize=31316 CPUtime=1182.33 cores=3
/proc/29571/stat : 29571 (lingeling) R 29567 29567 26347 0 -1 4202496 97113 0 0 0 118219 14 0 0 25 0 1 0 516995784 32067584 7026 33554432000 4194304 4352236 140735996121328 18446744073709551615 4328100 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/29571/statm: 7829 7026 127 39 0 6909 0
[pid=29572] ppid=29567 vsize=114440 CPUtime=587.42 cores=7
/proc/29572/stat : 29572 (march_hi) R 29567 29567 26347 0 -1 4202496 19595 0 0 0 58729 13 0 0 25 0 1 0 516995784 117186560 19488 33554432000 4194304 4331988 140736405780256 18446744073709551615 4229432 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29572/statm: 28610 19488 173 34 0 27052 0
Current children cumulated CPU time (s) 4722.86
Current children cumulated vsize (KiB) 2515944



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 5.17 5.13 5.09 6/181 29605
/proc/meminfo: memFree=30814956/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=1850012 CPUtime=1201.72 cores=1
/proc/29568/stat : 29568 (cryptominisat) R 29567 29567 26347 0 -1 4202496 103041 0 0 0 120135 37 0 0 25 0 1 0 516995784 1894412288 24481 33554432000 4194304 4587354 140736659967248 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/29568/statm: 462503 24481 337 96 0 459065 0
[pid=29569] ppid=29567 vsize=46060 CPUtime=1201.73 cores=5
/proc/29569/stat : 29569 (clasp) R 29567 29567 26347 0 -1 4202496 8457 0 0 0 120171 2 0 0 25 0 1 0 516995784 47165440 8344 33554432000 4194304 4785740 140736907391888 18446744073709551615 4553423 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/29569/statm: 11515 8344 341 145 0 8560 0
[pid=29570] ppid=29567 vsize=455960 CPUtime=598.06 cores=7
/proc/29570/stat : 29570 (TNM) R 29567 29567 26347 0 -1 4202496 355 0 0 0 59806 0 0 0 25 0 1 0 516995784 466903040 252 33554432000 134512640 134534564 4290972672 18446744073709551615 134529651 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29570/statm: 113990 252 95 6 0 112656 0
[pid=29571] ppid=29567 vsize=33108 CPUtime=1201.73 cores=3
/proc/29571/stat : 29571 (lingeling) R 29567 29567 26347 0 -1 4202496 97561 0 0 0 120159 14 0 0 25 0 1 0 516995784 33902592 7474 33554432000 4194304 4352236 140735996121328 18446744073709551615 4328009 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/29571/statm: 8277 7474 127 39 0 7357 0
[pid=29572] ppid=29567 vsize=114440 CPUtime=597.11 cores=7
/proc/29572/stat : 29572 (march_hi) R 29567 29567 26347 0 -1 4202496 19891 0 0 0 59697 14 0 0 25 0 1 0 516995784 117186560 19784 33554432000 4194304 4331988 140736405780256 18446744073709551615 4211509 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29572/statm: 28610 19784 173 34 0 27052 0
Current children cumulated CPU time (s) 4800.35
Current children cumulated vsize (KiB) 2517904

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 5.17 5.13 5.09 6/181 29605
/proc/meminfo: memFree=30814956/32951124 swapFree=67111344/67111528
[pid=29567] ppid=29565 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/29567/stat : 29567 (ppfolio) S 29565 29567 26347 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 516995784 18763776 389 33554432000 4194304 4287625 140737004821552 18446744073709551615 271550557711 0 0 4096 65536 18306454778076463103 0 0 17 1 0 0 0
/proc/29567/statm: 4581 389 341 23 0 91 0
[pid=29568] ppid=29567 vsize=1850012 CPUtime=1201.72 cores=1
/proc/29568/stat : 29568 (cryptominisat) R 29567 29567 26347 0 -1 4202496 103041 0 0 0 120135 37 0 0 25 0 1 0 516995784 1894412288 24481 33554432000 4194304 4587354 140736659967248 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/29568/statm: 462503 24481 337 96 0 459065 0
[pid=29569] ppid=29567 vsize=46060 CPUtime=1201.73 cores=5
/proc/29569/stat : 29569 (clasp) R 29567 29567 26347 0 -1 4202496 8457 0 0 0 120171 2 0 0 25 0 1 0 516995784 47165440 8344 33554432000 4194304 4785740 140736907391888 18446744073709551615 4553423 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/29569/statm: 11515 8344 341 145 0 8560 0
[pid=29570] ppid=29567 vsize=455960 CPUtime=598.06 cores=7
/proc/29570/stat : 29570 (TNM) R 29567 29567 26347 0 -1 4202496 355 0 0 0 59806 0 0 0 25 0 1 0 516995784 466903040 252 33554432000 134512640 134534564 4290972672 18446744073709551615 134529651 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29570/statm: 113990 252 95 6 0 112656 0
[pid=29571] ppid=29567 vsize=33108 CPUtime=1201.73 cores=3
/proc/29571/stat : 29571 (lingeling) R 29567 29567 26347 0 -1 4202496 97561 0 0 0 120159 14 0 0 25 0 1 0 516995784 33902592 7474 33554432000 4194304 4352236 140735996121328 18446744073709551615 4328009 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/29571/statm: 8277 7474 127 39 0 7357 0
[pid=29572] ppid=29567 vsize=114440 CPUtime=597.11 cores=7
/proc/29572/stat : 29572 (march_hi) R 29567 29567 26347 0 -1 4202496 19891 0 0 0 59697 14 0 0 25 0 1 0 516995784 117186560 19784 33554432000 4194304 4331988 140736405780256 18446744073709551615 4211509 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/29572/statm: 28610 19784 173 34 0 27052 0
Current children cumulated CPU time (s) 4800.35
Current children cumulated vsize (KiB) 2517904

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 29567 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=1999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=3999
# CPU time returned by wait4() is 0.005998
# while last known CPU time is 4800.35
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.35
CPU user time (s): 4799.68
CPU system time (s): 0.67
CPU usage (%): 399.459
Max. virtual memory (cumulated for all children) (KiB): 4241980

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.001999
system time used= 0.003999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 519
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 444
involuntary context switches= 6

runsolver used 1.9527 second user time and 4.6043 second system time

The end

Launcher Data 

Begin job on node133 at 2011-04-29 07:12:59
IDJOB=3191037
IDBENCH=82768
IDSOLVER=1612
FILE ID=node133/3191037-1304053979
RUNJOBID= node133-1304048754-28974
PBS_JOBID= 13170319
Free space on /tmp= 71896 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v600-c2556-S1245124682-071.UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3191037-1304053979/watcher-3191037-1304053979 -o /tmp/evaluation-result-3191037-1304053979/solver-3191037-1304053979 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3191037-1304053979.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b386c0165fb504493e3604d6c778ca42
RANDOM SEED=32375395

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31054784 kB
Buffers:        708400 kB
Cached:         767392 kB
SwapCached:        124 kB
Active:         554260 kB
Inactive:       984788 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31054784 kB
SwapTotal:    67111528 kB
SwapFree:     67111344 kB
Dirty:           10108 kB
Writeback:          24 kB
AnonPages:       63264 kB
Mapped:          14596 kB
Slab:           293748 kB
PageTables:       4536 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   184068 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71888 MiB
End job on node133 at 2011-04-29 07:33:03