Trace number 3317790

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-industrial? (MO) 1359.05 928.27

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-city-sequential-25nodes-1000size-3degree-100mindistance-3trucks-10packages-2008seed.060-SAT.cnf
MD5SUM0176f3dd09fed1b561c875e1d6538c8d
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark133.041
Satisfiable
(Un)Satisfiability was proved
Number of variables723130
Number of clauses3869060
Sum of the clauses size9210860
Maximum clause length502
Minimum clause length1
Number of clauses of size 1380
Number of clauses of size 23824280
Number of clauses of size 31440
Number of clauses of size 41440
Number of clauses of size 5360
Number of clauses of size over 541160

Solver Data 

0.30/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.30/0.47	c INFO: solving HOME/instance-3317790-1304268977.cnf
0.30/0.47	c INFO: computing task features
16.18/16.37	c INFO: computed variable-clause graph statistics
16.67/16.82	c INFO: computed clause balance statistics
69.98/70.19	c INFO: computed variable balance statistics
108.08/108.22	c INFO: computed small-clause counts
110.88/111.07	c INFO: computed Horn-clause counts
711.80/711.93	c INFO: computed variable graph statistics
711.80/711.94	c INFO: collected features for HOME/instance-3317790-1304268977.cnf in 711.04 s
711.80/711.96	c INFO: running cryptominisat-2.9.0@0 for 1164 with 4088 remaining (b = 0.58)
711.80/711.99	c INFO: running lingeling-276@0 for 485 with 4088 remaining (b = 0.35)
711.90/712.02	c INFO: running lingeling-276@0 for 97 with 4088 remaining (b = 0.16)
711.90/712.04	c INFO: running cryptominisat-2.9.0@0 for 1940 with 4088 remaining (b = 0.55)
1011.79/812.21	c INFO: running precosat-570@0 for 97 with 3988 remaining (b = 0.04)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317790-1304268977/watcher-3317790-1304268977 -o /tmp/evaluation-result-3317790-1304268977/solver-3317790-1304268977 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317790-1304268977.cnf -seed 1199177234 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.15 6.37 5.94 3/189 21445
/proc/meminfo: memFree=28734124/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=15384 CPUtime=0 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 800 0 0 0 0 0 0 0 22 0 1 0 538495834 15753216 685 33554432000 4194304 5683644 140734549798928 18446744073709551615 5287856 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/21445/statm: 3846 692 330 364 0 333 0

[startup+0.0713789 s]
/proc/loadavg: 5.15 6.37 5.94 3/189 21445
/proc/meminfo: memFree=28734124/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=54536 CPUtime=0.05 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 2043 0 0 0 4 1 0 0 17 0 1 0 538495834 55844864 1801 33554432000 4194304 5683644 140734549798928 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/21445/statm: 13634 1802 492 364 0 1284 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 54536

[startup+0.100397 s]
/proc/loadavg: 5.15 6.37 5.94 3/189 21445
/proc/meminfo: memFree=28734124/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=102012 CPUtime=0.06 cores=1,3,5,7
/proc/21445/stat : 21445 (python) S 21443 21445 21052 0 -1 4202496 3028 0 0 0 5 1 0 0 17 0 1 0 538495834 104460288 2718 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550535680 0 2147483391 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 25503 2718 841 364 0 1803 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 102012

[startup+0.300363 s]
/proc/loadavg: 5.15 6.37 5.94 3/189 21445
/proc/meminfo: memFree=28734124/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=163652 CPUtime=0.22 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 6647 931 1 0 17 5 0 0 17 0 1 0 538495834 167579648 4742 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562366272 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/21445/statm: 40913 4749 1303 364 0 3454 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 163652

[startup+0.700263 s]
/proc/loadavg: 5.15 6.37 5.94 3/189 21445
/proc/meminfo: memFree=28734124/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=365120 CPUtime=0.58 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 30842 931 1 0 46 12 0 0 17 0 1 0 538495834 373882880 28518 33554432000 4194304 5683644 140734549798928 18446744073709551615 4619779 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/21445/statm: 91280 28519 1761 364 0 27204 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 365120

[startup+1.50112 s]
/proc/loadavg: 5.15 6.37 5.94 4/190 21450
/proc/meminfo: memFree=28604164/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=421448 CPUtime=1.38 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 44629 931 1 0 125 13 0 0 18 0 1 0 538495834 431562752 42305 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550198469 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/21445/statm: 105362 42305 1761 364 0 41286 0
Current children cumulated CPU time (s) 1.38
Current children cumulated vsize (KiB) 421448

[startup+3.10083 s]
/proc/loadavg: 4.98 6.32 5.92 4/190 21450
/proc/meminfo: memFree=28527656/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=531236 CPUtime=2.98 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 71621 931 1 0 280 18 0 0 21 0 1 0 538495834 543985664 69297 33554432000 4194304 5683644 140734549798928 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 132809 69298 1761 364 0 68733 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 531236

[startup+6.30024 s]
/proc/loadavg: 4.98 6.32 5.92 4/190 21450
/proc/meminfo: memFree=28301704/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=755452 CPUtime=6.17 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 125828 931 1 0 592 25 0 0 25 0 1 0 538495834 773582848 123504 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550198469 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 188863 123505 1761 364 0 124787 0
Current children cumulated CPU time (s) 6.17
Current children cumulated vsize (KiB) 755452

[startup+12.7011 s]
/proc/loadavg: 4.75 6.22 5.90 4/190 21450
/proc/meminfo: memFree=27923960/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=1069464 CPUtime=12.58 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 296961 931 1 0 1214 44 0 0 25 0 1 0 538495834 1095131136 204739 33554432000 4194304 5683644 140734549798928 18446744073709551615 47636132824673 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 267366 204739 1814 364 0 203290 0
Current children cumulated CPU time (s) 12.58
Current children cumulated vsize (KiB) 1069464

[startup+25.5007 s]
/proc/loadavg: 4.48 6.12 5.87 4/190 21450
/proc/meminfo: memFree=28046908/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=918328 CPUtime=25.38 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 327138 931 1 0 2490 48 0 0 25 0 1 0 538495834 940367872 166999 33554432000 4194304 5683644 140734549798928 18446744073709551615 271553946293 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 229582 166999 1826 364 0 165506 0
Current children cumulated CPU time (s) 25.38
Current children cumulated vsize (KiB) 918328

[startup+51.1011 s]
/proc/loadavg: 3.97 5.87 5.79 4/190 21451
/proc/meminfo: memFree=28036336/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=918328 CPUtime=50.98 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 327138 931 1 0 5050 48 0 0 25 0 1 0 538495834 940367872 166999 33554432000 4194304 5683644 140734549798928 18446744073709551615 47636135280291 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 229582 166999 1826 364 0 165506 0
Current children cumulated CPU time (s) 50.98
Current children cumulated vsize (KiB) 918328

[startup+102.307 s]
/proc/loadavg: 3.44 5.40 5.63 4/190 21452
/proc/meminfo: memFree=28037644/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=918328 CPUtime=102.19 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 327139 931 1 0 10171 48 0 0 25 0 1 0 538495834 940367872 167000 33554432000 4194304 5683644 140734549798928 18446744073709551615 47636135188293 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 229582 167000 1827 364 0 165506 0
Current children cumulated CPU time (s) 102.19
Current children cumulated vsize (KiB) 918328

[startup+162.301 s]
/proc/loadavg: 4.11 5.25 5.57 4/190 21498
/proc/meminfo: memFree=27558480/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=948556 CPUtime=162.18 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 357340 931 1 0 16167 51 0 0 25 0 1 0 538495834 971321344 174563 33554432000 4194304 5683644 140734549798928 18446744073709551615 47636132411532 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 237139 174563 1833 364 0 173063 0
Current children cumulated CPU time (s) 162.18
Current children cumulated vsize (KiB) 948556

[startup+222.301 s]
/proc/loadavg: 3.44 4.83 5.39 4/190 21499
/proc/meminfo: memFree=27493972/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=948556 CPUtime=222.18 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 357340 931 1 0 22167 51 0 0 25 0 1 0 538495834 971321344 174563 33554432000 4194304 5683644 140734549798928 18446744073709551615 47636132411532 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 237139 174563 1833 364 0 173063 0
Current children cumulated CPU time (s) 222.18
Current children cumulated vsize (KiB) 948556

[startup+282.301 s]
/proc/loadavg: 3.17 4.51 5.25 4/190 21510
/proc/meminfo: memFree=27502868/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=948556 CPUtime=282.19 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 357340 931 1 0 28168 51 0 0 25 0 1 0 538495834 971321344 174563 33554432000 4194304 5683644 140734549798928 18446744073709551615 47636132411532 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 237139 174563 1833 364 0 173063 0
Current children cumulated CPU time (s) 282.19
Current children cumulated vsize (KiB) 948556

[startup+342.301 s]
/proc/loadavg: 3.06 4.22 5.10 4/190 21512
/proc/meminfo: memFree=27490900/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=948556 CPUtime=342.19 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 357340 931 1 0 34168 51 0 0 25 0 1 0 538495834 971321344 174563 33554432000 4194304 5683644 140734549798928 18446744073709551615 47636132411532 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 237139 174563 1833 364 0 173063 0
Current children cumulated CPU time (s) 342.19
Current children cumulated vsize (KiB) 948556

[startup+402.3 s]
/proc/loadavg: 3.02 4.01 4.97 7/202 21525
/proc/meminfo: memFree=27241168/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=948556 CPUtime=402.19 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 357340 931 1 0 40168 51 0 0 25 0 1 0 538495834 971321344 174563 33554432000 4194304 5683644 140734549798928 18446744073709551615 47636132411532 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 237139 174563 1833 364 0 173063 0
Current children cumulated CPU time (s) 402.19
Current children cumulated vsize (KiB) 948556

[startup+462.3 s]
/proc/loadavg: 4.99 4.40 5.04 7/202 21527
/proc/meminfo: memFree=26039072/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=948556 CPUtime=462.19 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 357340 931 1 0 46168 51 0 0 25 0 1 0 538495834 971321344 174563 33554432000 4194304 5683644 140734549798928 18446744073709551615 47636132411520 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 237139 174563 1833 364 0 173063 0
Current children cumulated CPU time (s) 462.19
Current children cumulated vsize (KiB) 948556

[startup+522.3 s]
/proc/loadavg: 3.79 4.16 4.92 4/190 21563
/proc/meminfo: memFree=28253776/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=948556 CPUtime=522.19 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 357340 931 1 0 52168 51 0 0 25 0 1 0 538495834 971321344 174563 33554432000 4194304 5683644 140734549798928 18446744073709551615 47636132411532 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 237139 174563 1833 364 0 173063 0
Current children cumulated CPU time (s) 522.19
Current children cumulated vsize (KiB) 948556

[startup+582.3 s]
/proc/loadavg: 3.30 3.94 4.79 4/190 21565
/proc/meminfo: memFree=28238176/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=948556 CPUtime=582.19 cores=1,3,5,7
/proc/21445/stat : 21445 (python) R 21443 21445 21052 0 -1 4202496 357340 931 1 0 58168 51 0 0 25 0 1 0 538495834 971321344 174563 33554432000 4194304 5683644 140734549798928 18446744073709551615 47636132411532 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21445/statm: 237139 174563 1833 364 0 173063 0
Current children cumulated CPU time (s) 582.19
Current children cumulated vsize (KiB) 948556

[startup+642.301 s]

################
# More data... #
################

/proc/21445/statm: 180969 105641 1884 364 0 116875 0
[pid=21585] ppid=21445 vsize=672608 CPUtime=0.27 cores=1,3,5,7
/proc/21585/stat : 21585 (python) S 21445 21445 21052 0 -1 4202560 1191 0 0 0 16 11 0 0 15 0 1 0 538567031 688750592 104119 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557711 0 0 16781312 514 10849611311281570056 0 0 17 5 0 0 0
/proc/21585/statm: 168152 104119 382 364 0 104070 0
[pid=21587] ppid=21585 vsize=2731320 CPUtime=109.68 cores=1,3,5,7
/proc/21587/stat : 21587 (cryptominisat-2) R 21585 21587 21587 0 -1 4202496 3004392 0 0 0 10588 380 0 0 25 0 1 0 538567032 2796871680 632445 33554432000 4194304 4677096 140736876105568 18446744073709551615 271550235021 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21587/statm: 682830 632445 390 118 0 677789 0
[pid=21589] ppid=21445 vsize=682864 CPUtime=0.15 cores=1,3,5,7
/proc/21589/stat : 21589 (python) S 21445 21445 21052 0 -1 4202560 953 0 0 0 7 8 0 0 18 0 1 0 538567034 699252736 104111 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557766 0 0 16781312 514 2573455343321317639 0 0 17 7 0 0 0
/proc/21589/statm: 170716 104111 370 364 0 106631 0
[pid=21591] ppid=21589 vsize=169896 CPUtime=55.32 cores=1,3,5,7
/proc/21591/stat : 21591 (lingeling) R 21589 21591 21591 0 -1 4202496 83368 0 0 0 5518 14 0 0 25 0 1 0 538567035 173973504 41644 33554432000 4194304 4352236 140735004812240 18446744073709551615 4278746 0 0 16781312 17506 0 0 0 17 7 0 0 0
/proc/21591/statm: 42474 41644 129 39 0 41554 0
[pid=21592] ppid=21445 vsize=693120 CPUtime=0.14 cores=1,3,5,7
/proc/21592/stat : 21592 (python) S 21445 21445 21052 0 -1 4202560 1100 0 0 0 7 7 0 0 19 0 2 0 538567036 709754880 104145 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562364363 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/21592/statm: 173280 104145 399 364 0 109192 0
[pid=21596] ppid=21592 vsize=165844 CPUtime=99.34 cores=1,3,5,7
/proc/21596/stat : 21596 (lingeling) T 21592 21596 21596 0 -1 4202496 100214 0 0 0 9918 16 0 0 25 0 1 0 538567040 169824256 40632 33554432000 4194304 4352236 140735855324608 18446744073709551615 4208575 0 0 16781312 17506 18446744071562691360 0 0 17 7 0 0 0
/proc/21596/statm: 41461 40632 129 39 0 40541 0
[pid=21592/tid=21600] ppid=21445 vsize=693120 CPUtime=0 cores=1,3,5,7
/proc/21592/task/21600/stat : 21600 (python) S 21445 21445 21052 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538577054 709754880 104145 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 514 18446604471133331904 0 0 -1 5 0 0 0
[pid=21594] ppid=21445 vsize=703376 CPUtime=0.16 cores=1,3,5,7
/proc/21594/stat : 21594 (python) S 21445 21445 21052 0 -1 4202560 985 0 0 0 7 9 0 0 15 0 1 0 538567039 720257024 104119 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557766 0 0 16781312 514 4606399570815975688 0 0 17 5 0 0 0
/proc/21594/statm: 175844 104119 370 364 0 111753 0
[pid=21597] ppid=21594 vsize=1046720 CPUtime=59.97 cores=1,3,5,7
/proc/21597/stat : 21597 (cryptominisat-2) R 21594 21597 21597 0 -1 4202496 240400 0 0 0 5963 34 0 0 25 0 1 0 538567041 1071841280 201570 33554432000 4194304 4677096 140735287167904 18446744073709551615 4336322 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/21597/statm: 261680 201570 387 118 0 256639 0
[pid=21601] ppid=21445 vsize=713632 CPUtime=0.01 cores=1,3,5,7
/proc/21601/stat : 21601 (python) S 21445 21445 21052 0 -1 4202560 935 0 0 0 0 1 0 0 18 0 1 0 538577056 730759168 104123 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557766 0 0 16781312 514 2573455343321317640 0 0 17 5 0 0 0
/proc/21601/statm: 178408 104123 370 364 0 114314 0
[pid=21603] ppid=21601 vsize=621812 CPUtime=5.01 cores=1,3,5,7
/proc/21603/stat : 21603 (precosat) R 21601 21603 21603 0 -1 4202496 163376 0 0 0 473 28 0 0 25 0 1 0 538577057 636735488 151810 33554432000 4194304 4890832 140734073302432 18446744073709551615 4294179 0 0 16781312 17442 0 0 0 17 7 0 0 0
/proc/21603/statm: 155453 151810 79 171 0 155280 0
[pid=21445/tid=21586] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21586/stat : 21586 (python) S 21443 21445 21052 0 -1 4202560 25 931 0 0 0 0 0 0 25 0 6 0 538567032 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 1 0 0 0
[pid=21445/tid=21590] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21590/stat : 21590 (python) S 21443 21445 21052 0 -1 4202560 25 931 0 0 0 0 0 0 20 0 6 0 538567035 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 7 0 0 0
[pid=21445/tid=21593] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21593/stat : 21593 (python) S 21443 21445 21052 0 -1 4202560 40 931 0 0 0 0 0 0 22 0 6 0 538567037 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 5 0 0 0
[pid=21445/tid=21595] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21595/stat : 21595 (python) S 21443 21445 21052 0 -1 4202560 20 931 0 0 0 0 0 0 25 0 6 0 538567040 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 5 0 0 0
[pid=21445/tid=21602] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21602/stat : 21602 (python) S 21443 21445 21052 0 -1 4202560 27 931 0 0 0 0 0 0 20 0 6 0 538577057 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1042.02
Current children cumulated vsize (KiB) 8925068

[startup+882.301 s]
/proc/loadavg: 8.93 6.65 5.67 10/222 21605
/proc/meminfo: memFree=19237992/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=723876 CPUtime=711.97 cores=1,3,5,7
/proc/21445/stat : 21445 (python) S 21443 21445 21052 0 -1 4202496 360862 931 1 0 71141 56 0 0 19 0 6 0 538495834 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562364363 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/21445/statm: 180969 105641 1884 364 0 116875 0
[pid=21585] ppid=21445 vsize=672608 CPUtime=0.44 cores=1,3,5,7
/proc/21585/stat : 21585 (python) S 21445 21445 21052 0 -1 4202560 1230 0 0 0 27 17 0 0 15 0 1 0 538567031 688750592 104119 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557711 0 0 16781312 514 10849611311281570056 0 0 17 5 0 0 0
/proc/21585/statm: 168152 104119 382 364 0 104070 0
[pid=21587] ppid=21585 vsize=4086856 CPUtime=142.98 cores=1,3,5,7
/proc/21587/stat : 21587 (cryptominisat-2) R 21585 21587 21587 0 -1 4202496 5896502 0 0 0 13533 765 0 0 25 0 1 0 538567032 4184940544 978859 33554432000 4194304 4677096 140736876105568 18446744073709551615 4365942 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21587/statm: 1021714 978859 390 118 0 1016673 0
[pid=21589] ppid=21445 vsize=682864 CPUtime=0.24 cores=1,3,5,7
/proc/21589/stat : 21589 (python) S 21445 21445 21052 0 -1 4202560 969 0 0 0 11 13 0 0 18 0 1 0 538567034 699252736 104111 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557766 0 0 16781312 514 2573455343321317639 0 0 17 7 0 0 0
/proc/21589/statm: 170716 104111 370 364 0 106631 0
[pid=21591] ppid=21589 vsize=177516 CPUtime=111.44 cores=1,3,5,7
/proc/21591/stat : 21591 (lingeling) R 21589 21591 21591 0 -1 4202496 103127 0 0 0 11128 16 0 0 25 0 1 0 538567035 181776384 43549 33554432000 4194304 4352236 140735004812240 18446744073709551615 4207627 0 0 16781312 17506 0 0 0 17 7 0 0 0
/proc/21591/statm: 44379 43549 129 39 0 43459 0
[pid=21592] ppid=21445 vsize=693120 CPUtime=0.14 cores=1,3,5,7
/proc/21592/stat : 21592 (python) S 21445 21445 21052 0 -1 4202560 1100 0 0 0 7 7 0 0 19 0 2 0 538567036 709754880 104145 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562364363 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/21592/statm: 173280 104145 399 364 0 109192 0
[pid=21596] ppid=21592 vsize=165844 CPUtime=99.34 cores=1,3,5,7
/proc/21596/stat : 21596 (lingeling) T 21592 21596 21596 0 -1 4202496 100214 0 0 0 9918 16 0 0 25 0 1 0 538567040 169824256 40632 33554432000 4194304 4352236 140735855324608 18446744073709551615 4208575 0 0 16781312 17506 18446744071562691360 0 0 17 7 0 0 0
/proc/21596/statm: 41461 40632 129 39 0 40541 0
[pid=21592/tid=21600] ppid=21445 vsize=693120 CPUtime=0 cores=1,3,5,7
/proc/21592/task/21600/stat : 21600 (python) S 21445 21445 21052 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538577054 709754880 104145 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 514 18446604471133331904 0 0 -1 5 0 0 0
[pid=21594] ppid=21445 vsize=703376 CPUtime=0.4 cores=1,3,5,7
/proc/21594/stat : 21594 (python) S 21445 21445 21052 0 -1 4202560 1140 0 0 0 23 17 0 0 15 0 1 0 538567039 720257024 104120 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557766 0 0 16781312 514 4606399570815975688 0 0 17 1 0 0 0
/proc/21594/statm: 175844 104120 371 364 0 111753 0
[pid=21597] ppid=21594 vsize=3239268 CPUtime=119.74 cores=1,3,5,7
/proc/21597/stat : 21597 (cryptominisat-2) R 21594 21597 21597 0 -1 4202496 4422767 0 0 0 11393 581 0 0 25 0 1 0 538567041 3317010432 758591 33554432000 4194304 4677096 140735287167904 18446744073709551615 4293950 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/21597/statm: 809817 758591 390 118 0 804776 0
[pid=21601] ppid=21445 vsize=713632 CPUtime=0.1 cores=1,3,5,7
/proc/21601/stat : 21601 (python) S 21445 21445 21052 0 -1 4202560 958 0 0 0 5 5 0 0 18 0 1 0 538577056 730759168 104123 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557766 0 0 16781312 514 2573455343321317640 0 0 17 5 0 0 0
/proc/21601/statm: 178408 104123 370 364 0 114314 0
[pid=21603] ppid=21601 vsize=721928 CPUtime=34.86 cores=1,3,5,7
/proc/21603/stat : 21603 (precosat) R 21601 21603 21603 0 -1 4202496 218668 0 0 0 3451 35 0 0 25 0 1 0 538577057 739254272 173202 33554432000 4194304 4890832 140734073302432 18446744073709551615 4295013 0 0 16781312 17442 0 0 0 17 5 0 0 0
/proc/21603/statm: 180482 173202 86 171 0 180309 0
[pid=21445/tid=21586] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21586/stat : 21586 (python) S 21443 21445 21052 0 -1 4202560 25 931 0 0 0 0 0 0 25 0 6 0 538567032 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 1 0 0 0
[pid=21445/tid=21590] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21590/stat : 21590 (python) S 21443 21445 21052 0 -1 4202560 25 931 0 0 0 0 0 0 20 0 6 0 538567035 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 7 0 0 0
[pid=21445/tid=21593] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21593/stat : 21593 (python) S 21443 21445 21052 0 -1 4202560 40 931 0 0 0 0 0 0 22 0 6 0 538567037 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 5 0 0 0
[pid=21445/tid=21595] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21595/stat : 21595 (python) S 21443 21445 21052 0 -1 4202560 20 931 0 0 0 0 0 0 25 0 6 0 538567040 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 5 0 0 0
[pid=21445/tid=21602] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21602/stat : 21602 (python) S 21443 21445 21052 0 -1 4202560 27 931 0 0 0 0 0 0 20 0 6 0 538577057 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1221.65
Current children cumulated vsize (KiB) 12580888



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+928.202 s]
/proc/loadavg: 9.07 7.01 5.83 10/222 21606
/proc/meminfo: memFree=15962616/32951124 swapFree=67111336/67111528
[pid=21445] ppid=21443 vsize=723876 CPUtime=711.97 cores=1,3,5,7
/proc/21445/stat : 21445 (python) S 21443 21445 21052 0 -1 4202496 360862 931 1 0 71141 56 0 0 19 0 6 0 538495834 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562364363 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/21445/statm: 180969 105641 1884 364 0 116875 0
[pid=21585] ppid=21445 vsize=672608 CPUtime=0.55 cores=1,3,5,7
/proc/21585/stat : 21585 (python) S 21445 21445 21052 0 -1 4202560 1250 0 0 0 34 21 0 0 15 0 1 0 538567031 688750592 104119 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557711 0 0 16781312 514 10849611311281570056 0 0 17 5 0 0 0
/proc/21585/statm: 168152 104119 382 364 0 104070 0
[pid=21587] ppid=21585 vsize=5246632 CPUtime=165.86 cores=1,3,5,7
/proc/21587/stat : 21587 (cryptominisat-2) R 21585 21587 21587 0 -1 4202496 7915077 0 0 0 15536 1050 0 0 25 0 1 0 538567032 5372551168 1260137 33554432000 4194304 4677096 140736876105568 18446744073709551615 4563753 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/21587/statm: 1311658 1260137 390 118 0 1306617 0
[pid=21589] ppid=21445 vsize=682864 CPUtime=0.31 cores=1,3,5,7
/proc/21589/stat : 21589 (python) S 21445 21445 21052 0 -1 4202560 978 0 0 0 15 16 0 0 18 0 1 0 538567034 699252736 104111 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557766 0 0 16781312 514 2573455343321317639 0 0 17 7 0 0 0
/proc/21589/statm: 170716 104111 370 364 0 106631 0
[pid=21591] ppid=21589 vsize=177276 CPUtime=156.98 cores=1,3,5,7
/proc/21591/stat : 21591 (lingeling) R 21589 21591 21591 0 -1 4202496 120931 0 0 0 15679 19 0 0 25 0 1 0 538567035 181530624 43489 33554432000 4194304 4352236 140735004812240 18446744073709551615 4234669 0 0 16781312 17506 0 0 0 17 7 0 0 0
/proc/21591/statm: 44319 43489 129 39 0 43399 0
[pid=21592] ppid=21445 vsize=693120 CPUtime=0.14 cores=1,3,5,7
/proc/21592/stat : 21592 (python) S 21445 21445 21052 0 -1 4202560 1100 0 0 0 7 7 0 0 19 0 2 0 538567036 709754880 104145 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562364363 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/21592/statm: 173280 104145 399 364 0 109192 0
[pid=21596] ppid=21592 vsize=165844 CPUtime=99.34 cores=1,3,5,7
/proc/21596/stat : 21596 (lingeling) T 21592 21596 21596 0 -1 4202496 100214 0 0 0 9918 16 0 0 25 0 1 0 538567040 169824256 40632 33554432000 4194304 4352236 140735855324608 18446744073709551615 4208575 0 0 16781312 17506 18446744071562691360 0 0 17 7 0 0 0
/proc/21596/statm: 41461 40632 129 39 0 40541 0
[pid=21592/tid=21600] ppid=21445 vsize=693120 CPUtime=0 cores=1,3,5,7
/proc/21592/task/21600/stat : 21600 (python) S 21445 21445 21052 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538577054 709754880 104145 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 514 18446604471133331904 0 0 -1 5 0 0 0
[pid=21594] ppid=21445 vsize=703376 CPUtime=0.57 cores=1,3,5,7
/proc/21594/stat : 21594 (python) S 21445 21445 21052 0 -1 4202560 1176 0 0 0 34 23 0 0 15 0 1 0 538567039 720257024 104120 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557766 0 0 16781312 514 4606399570815975688 0 0 17 1 0 0 0
/proc/21594/statm: 175844 104120 371 364 0 111753 0
[pid=21597] ppid=21594 vsize=5338260 CPUtime=165.47 cores=1,3,5,7
/proc/21597/stat : 21597 (cryptominisat-2) R 21594 21597 21597 0 -1 4202496 8372159 0 0 0 15441 1106 0 0 25 0 1 0 538567041 5466378240 1284392 33554432000 4194304 4677096 140735287167904 18446744073709551615 4300176 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/21597/statm: 1334565 1284392 390 118 0 1329524 0
[pid=21601] ppid=21445 vsize=713632 CPUtime=0.16 cores=1,3,5,7
/proc/21601/stat : 21601 (python) S 21445 21445 21052 0 -1 4202560 973 0 0 0 9 7 0 0 18 0 1 0 538577056 730759168 104123 33554432000 4194304 5683644 140734549798928 18446744073709551615 271550557766 0 0 16781312 514 2573455343321317640 0 0 17 5 0 0 0
/proc/21601/statm: 178408 104123 370 364 0 114314 0
[pid=21603] ppid=21601 vsize=755828 CPUtime=57.7 cores=1,3,5,7
/proc/21603/stat : 21603 (precosat) R 21601 21603 21603 0 -1 4202496 235624 0 0 0 5733 37 0 0 25 0 1 0 538577057 773967872 181683 33554432000 4194304 4890832 140734073302432 18446744073709551615 4216298 0 0 16781312 17442 0 0 0 17 5 0 0 0
/proc/21603/statm: 188957 181683 86 171 0 188784 0
[pid=21445/tid=21586] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21586/stat : 21586 (python) S 21443 21445 21052 0 -1 4202560 25 931 0 0 0 0 0 0 25 0 6 0 538567032 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 1 0 0 0
[pid=21445/tid=21590] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21590/stat : 21590 (python) S 21443 21445 21052 0 -1 4202560 25 931 0 0 0 0 0 0 20 0 6 0 538567035 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 7 0 0 0
[pid=21445/tid=21593] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21593/stat : 21593 (python) S 21443 21445 21052 0 -1 4202560 40 931 0 0 0 0 0 0 22 0 6 0 538567037 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 5 0 0 0
[pid=21445/tid=21595] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21595/stat : 21595 (python) S 21443 21445 21052 0 -1 4202560 20 931 0 0 0 0 0 0 25 0 6 0 538567040 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 5 0 0 0
[pid=21445/tid=21602] ppid=21443 vsize=723876 CPUtime=0 cores=1,3,5,7
/proc/21445/task/21602/stat : 21602 (python) S 21443 21445 21052 0 -1 4202560 27 931 0 0 0 0 0 0 20 0 6 0 538577057 741249024 105641 33554432000 4194304 5683644 140734549798928 18446744073709551615 271562361265 0 0 16781312 2 18446604468682726464 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1359.05
Current children cumulated vsize (KiB) 15873316

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 21445 and gives
#  childrusage.ru_utime.tv_sec=711
#  childrusage.ru_utime.tv_usec=416848
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=600908
# CPU time returned by wait4() is 712.018
# while last known CPU time is 1359.05
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 928.27
CPU time (s): 1359.05
CPU user time (s): 1335.47
CPU system time (s): 23.58
CPU usage (%): 146.407
Max. virtual memory (cumulated for all children) (KiB): 15873316

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 711.417
system time used= 0.600908
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 361795
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1051
involuntary context switches= 8945

runsolver used 1.54476 second user time and 3.73443 second system time

The end

Launcher Data 

Begin job on node133 at 2011-05-01 18:56:19
IDJOB=3317790
IDBENCH=82411
IDSOLVER=1706
FILE ID=node133/3317790-1304268977
RUNJOBID= node133-1304267282-21071
PBS_JOBID= 13173946
Free space on /tmp= 72744 MiB

SOLVER NAME= borg-sat 11.03.02-industrial
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-city-sequential-25nodes-1000size-3degree-100mindistance-3trucks-10packages-2008seed.060-SAT.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317790-1304268977/watcher-3317790-1304268977 -o /tmp/evaluation-result-3317790-1304268977/solver-3317790-1304268977 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317790-1304268977.cnf -seed 1199177234 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 0176f3dd09fed1b561c875e1d6538c8d
RANDOM SEED=1199177234

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      28734280 kB
Buffers:        569840 kB
Cached:        1443352 kB
SwapCached:        124 kB
Active:        2141372 kB
Inactive:      1719992 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      28734280 kB
SwapTotal:    67111528 kB
SwapFree:     67111336 kB
Dirty:          156652 kB
Writeback:           0 kB
AnonPages:     1847828 kB
Mapped:          22552 kB
Slab:           286464 kB
PageTables:       9388 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  2192360 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72744 MiB
End job on node133 at 2011-05-01 19:11:50