Trace number 3695967

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pb2sat 2012-05-19? (TO) 1800.1 1800.51

General information on the benchmark

Name/OPT-BIGINT-LIN/aries-da_nrp/
normalized-aries-da_network_20_2__17_12__256.opb
MD5SUMa048c41e885d13c7000ff4a85d8352bd
Bench CategoryOPT-BIGINT-LIN (optimisation, big integers, linear constraints)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark46877
Best CPU time to get the best result obtained on this benchmark3.3045
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables10258
Total number of constraints60
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)42
Number of constraints which are nor clauses,nor cardinality constraints18
Minimum length of a constraint256
Maximum length of a constraint2049
Number of terms in the objective function 10240
Biggest coefficient in the objective function 94409
Number of bits for the biggest coefficient in the objective function 17
Sum of the numbers in the objective function 446607872
Number of bits of the sum of numbers in the objective function 29
Biggest number in a constraint 94409
Number of bits of the biggest number in a constraint 17
Biggest sum of numbers in a constraint 446607872
Number of bits of the biggest sum of numbers29
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.17	c model valid
0.00/0.17	c current solution ok
0.00/0.48	o 212091
0.00/0.48	c current bound: 106045
0.00/0.48	c fallback to adder encoding
1384.82/1385.17	c model valid
1384.82/1385.17	c current solution ok
1384.82/1385.19	o 46877
1384.82/1385.19	c current bound: 23438
1384.82/1385.19	c fallback to adder encoding
1385.19/1385.98	c current bound: 35158
1385.19/1385.98	c fallback to adder encoding
1397.20/1397.53	c current bound: 41018
1397.20/1397.53	c fallback to adder encoding
1423.09/1423.42	c current bound: 43948
1423.09/1423.42	c fallback to adder encoding
1504.97/1505.32	c current bound: 45413
1504.97/1505.32	c fallback to adder encoding
1616.65/1617.06	c current bound: 46145
1616.65/1617.06	c fallback to adder encoding
1733.61/1734.07	c current bound: 46511
1733.61/1734.07	c fallback to adder encoding
1796.60/1797.06	c current bound: 46694
1796.60/1797.06	c fallback to adder encoding
1800.10/1800.51	c error: unexpected solver exit code
1800.10/1800.51	s UNKNOWN

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3695967-1338088521/watcher-3695967-1338088521 -o /tmp/evaluation-result-3695967-1338088521/solver-3695967-1338088521 -C 1800 -W 1900 -M 15500 HOME/runPb2sat.sh HOME/instance-3695967-1338088521.opb 1462386595 HOME HOME 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.92 1.09 1.48 3/174 5773
/proc/meminfo: memFree=31059084/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0

[startup+0.0496279 s]
/proc/loadavg: 0.92 1.09 1.48 3/174 5773
/proc/meminfo: memFree=31059084/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.100172 s]
/proc/loadavg: 0.92 1.09 1.48 3/174 5773
/proc/meminfo: memFree=31059084/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.401789 s]
/proc/loadavg: 0.92 1.09 1.48 3/174 5773
/proc/meminfo: memFree=31059084/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.700265 s]
/proc/loadavg: 0.92 1.09 1.48 3/174 5773
/proc/meminfo: memFree=31059084/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+1.50036 s]
/proc/loadavg: 0.92 1.09 1.48 3/181 5789
/proc/meminfo: memFree=30961596/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=149616 CPUtime=1.47 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) R 5773 5773 5734 0 -1 4202496 8729 5385 0 0 106 34 6 1 20 0 1 0 274978134 153206784 36256 33554432000 4194304 5873084 140737198440944 140737198435336 5129975 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 37404 36256 180 410 0 36986 0
Current children cumulated CPU time (s) 1.47
Current children cumulated vsize (KiB) 158764

[startup+3.10026 s]
/proc/loadavg: 0.92 1.09 1.48 3/183 5791
/proc/meminfo: memFree=30717048/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=87328 CPUtime=1.49 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 8730 5385 0 0 106 36 6 1 20 0 1 0 274978134 89423872 20685 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 21832 20685 180 410 0 21414 0
[pid=5790] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5790/stat : 5790 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 597 0 0 0 0 0 0 0 20 0 1 0 274978285 9367552 267 33554432000 4194304 5064252 140735840483840 140735840482752 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5790/statm: 2287 267 226 213 0 72 0
[pid=5791] ppid=5790 vsize=190956 CPUtime=1.57 cores=1,3,5,7
/proc/5791/stat : 5791 (glucose_static) R 5790 5773 5734 0 -1 4202496 37106 0 0 0 147 10 0 0 20 0 1 0 274978286 195538944 44207 33554432000 4194304 4896791 140735980642352 140735979591144 4223858 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/5791/statm: 47739 44207 78 172 0 47565 0
Current children cumulated CPU time (s) 3.06
Current children cumulated vsize (KiB) 296580

[startup+6.30026 s]
/proc/loadavg: 1.01 1.11 1.48 2/182 5825
/proc/meminfo: memFree=30713696/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=87328 CPUtime=1.49 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 8730 5385 0 0 106 36 6 1 20 0 1 0 274978134 89423872 20685 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 21832 20685 180 410 0 21414 0
[pid=5790] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5790/stat : 5790 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 597 0 0 0 0 0 0 0 20 0 1 0 274978285 9367552 267 33554432000 4194304 5064252 140735840483840 140735840482752 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5790/statm: 2287 267 226 213 0 72 0
[pid=5791] ppid=5790 vsize=194720 CPUtime=4.77 cores=1,3,5,7
/proc/5791/stat : 5791 (glucose_static) R 5790 5773 5734 0 -1 4202496 38420 0 0 0 466 11 0 0 20 0 1 0 274978286 199393280 45652 33554432000 4194304 4896791 140735980642352 140735980639560 4225151 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/5791/statm: 48680 45652 78 172 0 48506 0
Current children cumulated CPU time (s) 6.26
Current children cumulated vsize (KiB) 300344

[startup+12.7005 s]
/proc/loadavg: 1.09 1.12 1.48 3/181 5882
/proc/meminfo: memFree=30675808/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=87328 CPUtime=1.49 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 8730 5385 0 0 106 36 6 1 20 0 1 0 274978134 89423872 20685 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 21832 20685 180 410 0 21414 0
[pid=5790] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5790/stat : 5790 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 597 0 0 0 0 0 0 0 20 0 1 0 274978285 9367552 267 33554432000 4194304 5064252 140735840483840 140735840482752 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5790/statm: 2287 267 226 213 0 72 0
[pid=5791] ppid=5790 vsize=195380 CPUtime=11.17 cores=1,3,5,7
/proc/5791/stat : 5791 (glucose_static) R 5790 5773 5734 0 -1 4202496 38571 0 0 0 1106 11 0 0 20 0 1 0 274978286 200069120 45803 33554432000 4194304 4896791 140735980642352 140735980639560 4224844 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/5791/statm: 48845 45803 78 172 0 48671 0
Current children cumulated CPU time (s) 12.66
Current children cumulated vsize (KiB) 301004

[startup+25.5067 s]
/proc/loadavg: 1.37 1.18 1.49 3/183 5885
/proc/meminfo: memFree=29416908/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=87328 CPUtime=1.49 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 8730 5385 0 0 106 36 6 1 20 0 1 0 274978134 89423872 20685 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 21832 20685 180 410 0 21414 0
[pid=5790] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5790/stat : 5790 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 597 0 0 0 0 0 0 0 20 0 1 0 274978285 9367552 267 33554432000 4194304 5064252 140735840483840 140735840482752 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5790/statm: 2287 267 226 213 0 72 0
[pid=5791] ppid=5790 vsize=196360 CPUtime=23.97 cores=1,3,5,7
/proc/5791/stat : 5791 (glucose_static) R 5790 5773 5734 0 -1 4202496 38821 0 0 0 2386 11 0 0 20 0 1 0 274978286 201072640 46053 33554432000 4194304 4896791 140735980642352 140735980639560 4224684 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/5791/statm: 49090 46053 78 172 0 48916 0
Current children cumulated CPU time (s) 25.46
Current children cumulated vsize (KiB) 301984

[startup+51.1004 s]
/proc/loadavg: 1.58 1.24 1.51 3/183 5885
/proc/meminfo: memFree=29424780/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=87328 CPUtime=1.49 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 8730 5385 0 0 106 36 6 1 20 0 1 0 274978134 89423872 20685 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 21832 20685 180 410 0 21414 0
[pid=5790] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5790/stat : 5790 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 597 0 0 0 0 0 0 0 20 0 1 0 274978285 9367552 267 33554432000 4194304 5064252 140735840483840 140735840482752 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5790/statm: 2287 267 226 213 0 72 0
[pid=5791] ppid=5790 vsize=197716 CPUtime=49.56 cores=1,3,5,7
/proc/5791/stat : 5791 (glucose_static) R 5790 5773 5734 0 -1 4202496 39549 0 0 0 4945 11 0 0 20 0 1 0 274978286 202461184 46912 33554432000 4194304 4896791 140735980642352 140735980639400 4207807 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/5791/statm: 49429 46912 78 172 0 49255 0
Current children cumulated CPU time (s) 51.05
Current children cumulated vsize (KiB) 303340

[startup+102.3 s]
/proc/loadavg: 1.82 1.36 1.53 3/183 5885
/proc/meminfo: memFree=29414404/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=87328 CPUtime=1.49 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 8730 5385 0 0 106 36 6 1 20 0 1 0 274978134 89423872 20685 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 21832 20685 180 410 0 21414 0
[pid=5790] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5790/stat : 5790 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 597 0 0 0 0 0 0 0 20 0 1 0 274978285 9367552 267 33554432000 4194304 5064252 140735840483840 140735840482752 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5790/statm: 2287 267 226 213 0 72 0
[pid=5791] ppid=5790 vsize=231976 CPUtime=100.74 cores=1,3,5,7
/proc/5791/stat : 5791 (glucose_static) R 5790 5773 5734 0 -1 4202496 43054 0 0 0 10061 13 0 0 20 0 1 0 274978286 237543424 49498 33554432000 4194304 4896791 140735980642352 140735980639560 4225008 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/5791/statm: 57994 49498 78 172 0 57820 0
Current children cumulated CPU time (s) 102.23
Current children cumulated vsize (KiB) 337600

[startup+162.3 s]
/proc/loadavg: 1.93 1.47 1.56 3/183 5885
/proc/meminfo: memFree=29411560/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=87328 CPUtime=1.49 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 8730 5385 0 0 106 36 6 1 20 0 1 0 274978134 89423872 20685 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 21832 20685 180 410 0 21414 0
[pid=5790] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5790/stat : 5790 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 597 0 0 0 0 0 0 0 20 0 1 0 274978285 9367552 267 33554432000 4194304 5064252 140735840483840 140735840482752 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5790/statm: 2287 267 226 213 0 72 0
[pid=5791] ppid=5790 vsize=232900 CPUtime=160.72 cores=1,3,5,7
/proc/5791/stat : 5791 (glucose_static) R 5790 5773 5734 0 -1 4202496 43322 0 0 0 16059 13 0 0 20 0 1 0 274978286 238489600 50270 33554432000 4194304 4896791 140735980642352 140735980639560 4224983 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/5791/statm: 58225 50270 78 172 0 58051 0
Current children cumulated CPU time (s) 162.21

################
# More data... #
################

/proc/5904/stat : 5904 (glucose_static) R 5903 5773 5734 0 -1 4202496 23337 0 0 0 2429 5 0 0 20 0 1 0 275117928 107905024 22018 33554432000 4194304 4896791 140734960101584 140734960098680 4203775 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/5904/statm: 26344 22018 78 172 0 26170 0
Current children cumulated CPU time (s) 1421.99
Current children cumulated vsize (KiB) 206032

[startup+1482.3 s]
/proc/loadavg: 2.05 1.99 1.86 3/184 5911
/proc/meminfo: memFree=29510264/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=82360 CPUtime=1423.56 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 26978 146094 0 0 247 51 141999 59 20 0 1 0 274978134 84336640 18931 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 20590 18931 180 410 0 20172 0
[pid=5906] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5906/stat : 5906 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 598 0 0 0 0 0 0 0 20 0 1 0 275120519 9367552 267 33554432000 4194304 5064252 140734253786256 140734253785168 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5906/statm: 2287 267 226 213 0 72 0
[pid=5907] ppid=5906 vsize=108924 CPUtime=58.41 cores=1,3,5,7
/proc/5907/stat : 5907 (glucose_static) R 5906 5773 5734 0 -1 4202496 25689 0 0 0 5834 7 0 0 20 0 1 0 275120519 111538176 24020 33554432000 4194304 4896791 140735628315280 140735628312488 4203775 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/5907/statm: 27231 24020 78 172 0 27057 0
Current children cumulated CPU time (s) 1481.97
Current children cumulated vsize (KiB) 209580

[startup+1542.3 s]
/proc/loadavg: 2.02 1.99 1.86 3/183 5914
/proc/meminfo: memFree=29502916/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=82360 CPUtime=1505.44 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 30785 173784 0 0 286 55 150134 69 20 0 1 0 274978134 84336640 18931 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 20590 18931 180 410 0 20172 0
[pid=5913] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5913/stat : 5913 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 598 0 0 0 0 0 0 0 20 0 1 0 275128709 9367552 267 33554432000 4194304 5064252 140733367485824 140733367484736 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5913/statm: 2287 267 226 213 0 72 0
[pid=5914] ppid=5913 vsize=105644 CPUtime=36.52 cores=1,3,5,7
/proc/5914/stat : 5914 (glucose_static) R 5913 5773 5734 0 -1 4202496 21821 0 0 0 3648 4 0 0 20 0 1 0 275128709 108179456 24112 33554432000 4194304 4896791 140735736113296 140735736110504 4224684 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/5914/statm: 26411 24112 78 172 0 26237 0
Current children cumulated CPU time (s) 1541.96
Current children cumulated vsize (KiB) 206300

[startup+1602.3 s]
/proc/loadavg: 2.00 1.99 1.87 3/183 5914
/proc/meminfo: memFree=29489520/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=82360 CPUtime=1505.44 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 30785 173784 0 0 286 55 150134 69 20 0 1 0 274978134 84336640 18931 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 20590 18931 180 410 0 20172 0
[pid=5913] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5913/stat : 5913 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 598 0 0 0 0 0 0 0 20 0 1 0 275128709 9367552 267 33554432000 4194304 5064252 140733367485824 140733367484736 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5913/statm: 2287 267 226 213 0 72 0
[pid=5914] ppid=5913 vsize=108556 CPUtime=96.51 cores=1,3,5,7
/proc/5914/stat : 5914 (glucose_static) R 5913 5773 5734 0 -1 4202496 23859 0 0 0 9646 5 0 0 20 0 1 0 275128709 111161344 25416 33554432000 4194304 4896791 140735736113296 140735736110504 4203749 0 0 4096 0 0 0 0 17 5 0 0 0 0 0
/proc/5914/statm: 27139 25416 78 172 0 26965 0
Current children cumulated CPU time (s) 1601.95
Current children cumulated vsize (KiB) 209212

[startup+1662.31 s]
/proc/loadavg: 2.00 1.99 1.88 3/183 5917
/proc/meminfo: memFree=29471564/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=82360 CPUtime=1617.2 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 35252 200826 0 0 329 60 161254 77 20 0 1 0 274978134 84336640 19443 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 20590 19443 180 410 0 20172 0
[pid=5916] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5916/stat : 5916 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 597 0 0 0 0 0 0 0 20 0 1 0 275139888 9367552 267 33554432000 4194304 5064252 140737236240032 140737236238944 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5916/statm: 2287 267 226 213 0 72 0
[pid=5917] ppid=5916 vsize=115288 CPUtime=44.73 cores=1,3,5,7
/proc/5917/stat : 5917 (glucose_static) R 5916 5773 5734 0 -1 4202496 22942 0 0 0 4468 5 0 0 20 0 1 0 275139888 118054912 26371 33554432000 4194304 4896791 140736691022224 140736691019432 4203775 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/5917/statm: 28822 26371 78 172 0 28648 0
Current children cumulated CPU time (s) 1661.93
Current children cumulated vsize (KiB) 215944

[startup+1722.3 s]
/proc/loadavg: 2.00 1.99 1.88 3/183 5917
/proc/meminfo: memFree=29470052/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=82360 CPUtime=1617.2 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 35252 200826 0 0 329 60 161254 77 20 0 1 0 274978134 84336640 19443 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 20590 19443 180 410 0 20172 0
[pid=5916] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5916/stat : 5916 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 597 0 0 0 0 0 0 0 20 0 1 0 275139888 9367552 267 33554432000 4194304 5064252 140737236240032 140737236238944 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5916/statm: 2287 267 226 213 0 72 0
[pid=5917] ppid=5916 vsize=117076 CPUtime=104.71 cores=1,3,5,7
/proc/5917/stat : 5917 (glucose_static) R 5916 5773 5734 0 -1 4202496 23932 0 0 0 10466 5 0 0 20 0 1 0 275139888 119885824 26725 33554432000 4194304 4896791 140736691022224 140736691019320 4209656 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/5917/statm: 29269 26725 78 172 0 29095 0
Current children cumulated CPU time (s) 1721.91
Current children cumulated vsize (KiB) 217732

[startup+1782.3 s]
/proc/loadavg: 2.00 1.99 1.89 3/182 5920
/proc/meminfo: memFree=29476476/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=82552 CPUtime=1734.18 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 39194 228675 0 0 372 65 172895 86 20 0 1 0 274978134 84533248 19491 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 20638 19491 180 410 0 20220 0
[pid=5919] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5919/stat : 5919 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 596 0 0 0 0 0 0 0 20 0 1 0 275151589 9367552 266 33554432000 4194304 5064252 140736281627616 140736281626528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5919/statm: 2287 266 226 213 0 72 0
[pid=5920] ppid=5919 vsize=118972 CPUtime=47.72 cores=1,3,5,7
/proc/5920/stat : 5920 (glucose_static) R 5919 5773 5734 0 -1 4202496 25699 0 0 0 4766 6 0 0 20 0 1 0 275151589 121827328 24566 33554432000 4194304 4896791 140733939801600 140733939798808 4204115 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/5920/statm: 29743 24566 78 172 0 29569 0
Current children cumulated CPU time (s) 1781.9
Current children cumulated vsize (KiB) 219820



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.5 s]
/proc/loadavg: 2.00 1.99 1.89 3/183 5924
/proc/meminfo: memFree=29478692/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=82552 CPUtime=1797.14 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 43134 256867 0 0 415 69 179135 95 20 0 1 0 274978134 84533248 19491 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 20638 19491 180 410 0 20220 0
[pid=5923] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5923/stat : 5923 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 596 0 0 0 0 0 0 0 20 0 1 0 275157887 9367552 267 33554432000 4194304 5064252 140734543874352 140734543873264 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5923/statm: 2287 267 226 213 0 72 0
[pid=5924] ppid=5923 vsize=112588 CPUtime=2.96 cores=1,3,5,7
/proc/5924/stat : 5924 (glucose_static) R 5923 5773 5734 0 -1 4202496 21147 0 0 0 291 5 0 0 20 0 1 0 275157887 115290112 23802 33554432000 4194304 4896791 140736906567792 140736906565000 4224983 0 0 4096 0 0 0 0 17 1 0 0 0 0 0
/proc/5924/statm: 28147 23802 78 172 0 27973 0
Current children cumulated CPU time (s) 1800.1
Current children cumulated vsize (KiB) 213436

Sending SIGTERM to process tree (bottom up)

Solver just ended. Dumping a history of the last processes samples

[startup+1800.5 s]
/proc/loadavg: 2.00 1.99 1.89 3/183 5924
/proc/meminfo: memFree=29478692/32873648 swapFree=6860/6860
[pid=5773] ppid=5771 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5773/stat : 5773 (runPb2sat.sh) S 5771 5773 5734 0 -1 4202496 358 0 0 0 0 0 0 0 20 0 1 0 274978134 9367552 275 33554432000 4194304 5064252 140733738409616 140733738408528 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/5773/statm: 2287 275 233 213 0 72 0
[pid=5774] ppid=5773 vsize=82552 CPUtime=1797.14 cores=1,3,5,7
/proc/5774/stat : 5774 (pb2sat) S 5773 5773 5734 0 -1 4202496 43134 256867 0 0 415 69 179135 95 20 0 1 0 274978134 84533248 19491 33554432000 4194304 5873084 140737198440944 140737198434760 5124602 0 65536 4102 0 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/5774/statm: 20638 19491 180 410 0 20220 0
[pid=5923] ppid=5774 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/5923/stat : 5923 (satsolver.sh) S 5774 5773 5734 0 -1 4202496 596 0 0 0 0 0 0 0 20 0 1 0 275157887 9367552 267 33554432000 4194304 5064252 140734543874352 140734543873264 266435454542 0 65536 4100 65538 18446744071579302229 0 0 17 3 0 0 0 0 0
/proc/5923/statm: 2287 267 226 213 0 72 0
[pid=5924] ppid=5923 vsize=112588 CPUtime=2.96 cores=1,3,5,7
/proc/5924/stat : 5924 (glucose_static) R 5923 5773 5734 0 -1 4202496 21147 0 0 0 291 5 0 0 20 0 1 0 275157887 115290112 23802 33554432000 4194304 4896791 140736906567792 140736906565000 4224983 0 0 4096 0 0 0 0 17 1 0 0 0 0 0
/proc/5924/statm: 28147 23802 78 172 0 27973 0
Current children cumulated CPU time (s) 1800.1
Current children cumulated vsize (KiB) 213436

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 5773 and gives
#  childrusage.ru_utime.tv_sec=1795
#  childrusage.ru_utime.tv_usec=513040
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=657747
# CPU time returned by wait4() is 1797.17
# while last known CPU time is 1800.1
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1800.51
CPU time (s): 1800.1
CPU user time (s): 1798.41
CPU system time (s): 1.69
CPU usage (%): 99.977
Max. virtual memory (cumulated for all children) (KiB): 457216

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1795.51
system time used= 1.65775
maximum resident set size= 295044
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 300977
page faults= 0
swaps= 0
block input operations= 0
block output operations= 606712
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 177
involuntary context switches= 2257

runsolver used 6.07608 second user time and 17.8683 second system time

The end

Launcher Data 

Begin job on node116 at 2012-05-27 05:15:21
IDJOB=3695967
IDBENCH=71888
IDSOLVER=2274
FILE ID=node116/3695967-1338088521
RUNJOBID= node116-1338088521-5751
PBS_JOBID= 14624463
Free space on /tmp= 71472 MiB

SOLVER NAME= pb2sat 2012-05-19
BENCH NAME= PB09/normalized-PB09/OPT-BIGINT-LIN/aries-da_nrp/normalized-aries-da_network_20_2__17_12__256.opb
COMMAND LINE= DIR/runPb2sat.sh BENCHNAME RANDOMSEED TMPDIR DIR
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3695967-1338088521/watcher-3695967-1338088521 -o /tmp/evaluation-result-3695967-1338088521/solver-3695967-1338088521 -C 1800 -W 1900 -M 15500  HOME/runPb2sat.sh HOME/instance-3695967-1338088521.opb 1462386595 HOME HOME

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= a048c41e885d13c7000ff4a85d8352bd
RANDOM SEED=1462386595

node116.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.556
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.11
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.556
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.556
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5438.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.556
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.556
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.556
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.556
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.556
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        31059348 kB
Buffers:          293424 kB
Cached:           685124 kB
SwapCached:         3244 kB
Active:           743320 kB
Inactive:         247004 kB
Active(anon):       2388 kB
Inactive(anon):     8932 kB
Active(file):     740932 kB
Inactive(file):   238072 kB
Unevictable:        6860 kB
Mlocked:            6860 kB
SwapTotal:      67108856 kB
SwapFree:       67103660 kB
Dirty:             17476 kB
Writeback:             0 kB
AnonPages:         13484 kB
Mapped:            11788 kB
Shmem:                96 kB
Slab:             683596 kB
SReclaimable:     127656 kB
SUnreclaim:       555940 kB
KernelStack:        1360 kB
PageTables:         4932 kB
NFS_Unstable:          4 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:     102684 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71128 MiB
End job on node116 at 2012-05-27 05:45:21