Trace number 3378761

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-industrial (fixed)? (MO) 1631.82 996.079

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.020-NOTKNOWN.cnf
MD5SUM5d1253e2dcc0e3e55992672756beadbc
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2074.86
Satisfiable
(Un)Satisfiability was proved
Number of variables756832
Number of clauses4116966
Sum of the clauses size9765566
Maximum clause length1178
Minimum clause length1
Number of clauses of size 1846
Number of clauses of size 24082840
Number of clauses of size 3640
Number of clauses of size 4800
Number of clauses of size 5800
Number of clauses of size over 531040

Solver Data 

0.40/0.93	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.40/0.93	c INFO: solving HOME/instance-3378761-1305465319.cnf
0.40/0.93	c INFO: computing task features
17.39/17.90	c INFO: computed variable-clause graph statistics
17.78/18.39	c INFO: computed clause balance statistics
74.99/75.54	c INFO: computed variable balance statistics
114.69/115.21	c INFO: computed small-clause counts
117.90/118.47	c INFO: computed Horn-clause counts
782.32/782.88	c INFO: computed variable graph statistics
782.32/782.90	c INFO: collected features for HOME/instance-3378761-1305465319.cnf in 781.01 s
782.42/782.91	c INFO: running lingeling-276@0 for 418 with 418 remaining (b = 0.35)
782.42/782.92	c INFO: running cryptominisat-2.9.0@0 for 418 with 418 remaining (b = 0.28)
782.42/782.94	c INFO: running lingeling-276@0 for 417 with 417 remaining (b = 0.23)
782.42/782.96	c INFO: running cryptominisat-2.9.0@0 for 417 with 417 remaining (b = 0.19)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378761-1305465319/watcher-3378761-1305465319 -o /tmp/evaluation-result-3378761-1305465319/solver-3378761-1305465319 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3378761-1305465319.cnf -seed 2051470122 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.16 0.03 0.54 4/397 26644
/proc/meminfo: memFree=18316332/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=15772 CPUtime=0 cores=1,3,5,7
/proc/26644/stat : 26644 (python) S 26642 26644 26612 0 -1 4202496 934 0 0 0 0 0 0 0 17 0 1 0 736762859 16150528 813 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415389696 0 2147483391 16781312 2 18446744071699199166 0 0 17 1 0 0 0
/proc/26644/statm: 3943 813 356 364 0 430 0

[startup+0.036802 s]
/proc/loadavg: 0.16 0.03 0.54 4/397 26644
/proc/meminfo: memFree=18316332/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=16808 CPUtime=0.01 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 1227 0 0 0 1 0 0 0 17 0 1 0 736762859 17211392 1085 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415389696 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 4202 1085 417 364 0 689 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16808

[startup+0.100792 s]
/proc/loadavg: 0.16 0.03 0.54 4/397 26644
/proc/meminfo: memFree=18316332/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=84600 CPUtime=0.06 cores=1,3,5,7
/proc/26644/stat : 26644 (python) S 26642 26644 26612 0 -1 4202496 2407 0 0 0 4 2 0 0 17 0 1 0 736762859 86630400 2135 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415389696 0 2147483391 16781312 2 18446744071699199166 0 0 17 1 0 0 0
/proc/26644/statm: 21150 2135 707 364 0 1367 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 84600

[startup+0.300758 s]
/proc/loadavg: 0.16 0.03 0.54 4/397 26644
/proc/meminfo: memFree=18316332/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=135744 CPUtime=0.2 cores=1,3,5,7
/proc/26644/stat : 26644 (python) S 26642 26644 26612 0 -1 4202496 5959 932 1 0 13 7 0 0 17 0 1 0 736762859 139001856 4149 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415387653 0 2147483391 16781312 2 18446744071699199166 0 0 17 1 0 0 0
/proc/26644/statm: 33936 4149 1032 364 0 3122 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 135744

[startup+0.700681 s]
/proc/loadavg: 0.16 0.03 0.54 4/397 26644
/proc/meminfo: memFree=18316332/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=234160 CPUtime=0.34 cores=1,3,5,7
/proc/26644/stat : 26644 (python) S 26642 26644 26612 0 -1 4202496 7860 932 1 0 20 14 0 0 15 0 1 0 736762859 239779840 5771 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415389696 0 2147483391 16781312 2 18446744071699199166 0 0 17 1 0 0 0
/proc/26644/statm: 58540 5771 1590 364 0 4339 0
Current children cumulated CPU time (s) 0.34
Current children cumulated vsize (KiB) 234160

[startup+1.50052 s]
/proc/loadavg: 0.16 0.03 0.54 3/398 26649
/proc/meminfo: memFree=18178448/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=394364 CPUtime=0.99 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 37919 932 1 0 78 21 0 0 16 0 1 0 736762859 403828736 35602 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415052485 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 98591 35604 1761 364 0 34515 0
Current children cumulated CPU time (s) 0.99
Current children cumulated vsize (KiB) 394364

[startup+3.10022 s]
/proc/loadavg: 0.16 0.03 0.54 3/398 26649
/proc/meminfo: memFree=18178448/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=503888 CPUtime=2.59 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 65359 932 1 0 233 26 0 0 18 0 1 0 736762859 515981312 63042 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415052485 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 125972 63043 1761 364 0 61896 0
Current children cumulated CPU time (s) 2.59
Current children cumulated vsize (KiB) 503888

[startup+6.30061 s]
/proc/loadavg: 0.47 0.10 0.55 3/401 26659
/proc/meminfo: memFree=17486808/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=728900 CPUtime=5.79 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 120336 932 1 0 541 38 0 0 25 0 1 0 736762859 746393600 118019 33554432000 4194304 5683644 140734000423296 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 182225 118019 1761 364 0 118149 0
Current children cumulated CPU time (s) 5.79
Current children cumulated vsize (KiB) 728900

[startup+12.7004 s]
/proc/loadavg: 0.59 0.13 0.56 3/401 26660
/proc/meminfo: memFree=16667900/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=1085856 CPUtime=12.19 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 354122 932 1 0 1151 68 0 0 25 0 1 0 736762859 1111916544 208266 33554432000 4194304 5683644 140734000423296 18446744073709551615 47839622923898 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 271464 208267 1814 364 0 207388 0
Current children cumulated CPU time (s) 12.19
Current children cumulated vsize (KiB) 1085856

[startup+25.501 s]
/proc/loadavg: 0.90 0.22 0.58 3/401 26660
/proc/meminfo: memFree=15973692/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=989364 CPUtime=24.99 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 539298 932 1 0 2411 88 0 0 25 0 1 0 736762859 1013108736 184758 33554432000 4194304 5683644 140734000423296 18446744073709551615 4618496 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 247341 184758 1826 364 0 183265 0
Current children cumulated CPU time (s) 24.99
Current children cumulated vsize (KiB) 989364

[startup+51.1011 s]
/proc/loadavg: 1.28 0.36 0.62 3/401 26661
/proc/meminfo: memFree=15985584/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=989364 CPUtime=50.59 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 539298 932 1 0 4971 88 0 0 25 0 1 0 736762859 1013108736 184758 33554432000 4194304 5683644 140734000423296 18446744073709551615 4562946 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 247341 184758 1826 364 0 183265 0
Current children cumulated CPU time (s) 50.59
Current children cumulated vsize (KiB) 989364

[startup+102.314 s]
/proc/loadavg: 1.68 0.61 0.69 3/401 26662
/proc/meminfo: memFree=16050864/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=989364 CPUtime=101.8 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 539299 932 1 0 10092 88 0 0 25 0 1 0 736762859 1013108736 184759 33554432000 4194304 5683644 140734000423296 18446744073709551615 47839622691330 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 247341 184759 1827 364 0 183265 0
Current children cumulated CPU time (s) 101.8
Current children cumulated vsize (KiB) 989364

[startup+162.301 s]
/proc/loadavg: 1.88 0.86 0.77 3/401 26664
/proc/meminfo: memFree=16050048/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=989364 CPUtime=161.8 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 747955 932 1 0 16066 114 0 0 25 0 1 0 736762859 1013108736 184765 33554432000 4194304 5683644 140734000423296 18446744073709551615 47839622510732 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 247341 184765 1833 364 0 183265 0
Current children cumulated CPU time (s) 161.8
Current children cumulated vsize (KiB) 989364

[startup+222.301 s]
/proc/loadavg: 1.95 1.07 0.84 3/401 26665
/proc/meminfo: memFree=16050988/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=989364 CPUtime=221.8 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 747955 932 1 0 22066 114 0 0 25 0 1 0 736762859 1013108736 184765 33554432000 4194304 5683644 140734000423296 18446744073709551615 47839622510732 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 247341 184765 1833 364 0 183265 0
Current children cumulated CPU time (s) 221.8
Current children cumulated vsize (KiB) 989364

[startup+282.3 s]
/proc/loadavg: 1.98 1.23 0.91 3/401 26667
/proc/meminfo: memFree=16051064/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=989364 CPUtime=281.8 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 747955 932 1 0 28066 114 0 0 25 0 1 0 736762859 1013108736 184765 33554432000 4194304 5683644 140734000423296 18446744073709551615 47839622510732 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 247341 184765 1833 364 0 183265 0
Current children cumulated CPU time (s) 281.8
Current children cumulated vsize (KiB) 989364

[startup+342.301 s]
/proc/loadavg: 2.03 1.38 0.98 3/401 26668
/proc/meminfo: memFree=16050904/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=989364 CPUtime=341.81 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 747955 932 1 0 34067 114 0 0 25 0 1 0 736762859 1013108736 184765 33554432000 4194304 5683644 140734000423296 18446744073709551615 47839622510732 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 247341 184765 1833 364 0 183265 0
Current children cumulated CPU time (s) 341.81
Current children cumulated vsize (KiB) 989364

[startup+402.3 s]
/proc/loadavg: 2.01 1.49 1.04 3/401 26670
/proc/meminfo: memFree=16051220/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=989364 CPUtime=401.81 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 747955 932 1 0 40067 114 0 0 25 0 1 0 736762859 1013108736 184765 33554432000 4194304 5683644 140734000423296 18446744073709551615 47839622510720 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 247341 184765 1833 364 0 183265 0
Current children cumulated CPU time (s) 401.81
Current children cumulated vsize (KiB) 989364

[startup+462.301 s]
/proc/loadavg: 2.06 1.60 1.11 3/401 26672
/proc/meminfo: memFree=16051164/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=989364 CPUtime=461.81 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 747955 932 1 0 46067 114 0 0 25 0 1 0 736762859 1013108736 184765 33554432000 4194304 5683644 140734000423296 18446744073709551615 47839622510732 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 247341 184765 1833 364 0 183265 0
Current children cumulated CPU time (s) 461.81
Current children cumulated vsize (KiB) 989364

[startup+522.301 s]
/proc/loadavg: 2.02 1.67 1.16 3/401 26673
/proc/meminfo: memFree=16051740/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=989364 CPUtime=521.81 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 747955 932 1 0 52067 114 0 0 25 0 1 0 736762859 1013108736 184765 33554432000 4194304 5683644 140734000423296 18446744073709551615 47839622510732 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 247341 184765 1833 364 0 183265 0
Current children cumulated CPU time (s) 521.81
Current children cumulated vsize (KiB) 989364

[startup+582.3 s]
/proc/loadavg: 2.01 1.73 1.21 3/401 26675
/proc/meminfo: memFree=16051560/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=989364 CPUtime=581.81 cores=1,3,5,7
/proc/26644/stat : 26644 (python) R 26642 26644 26612 0 -1 4202496 747955 932 1 0 58067 114 0 0 25 0 1 0 736762859 1013108736 184765 33554432000 4194304 5683644 140734000423296 18446744073709551615 47839622510720 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26644/statm: 247341 184765 1833 364 0 183265 0
Current children cumulated CPU time (s) 581.81
Current children cumulated vsize (KiB) 989364

[startup+642.301 s]

################
# More data... #
################

/proc/26686/stat : 26686 (cryptominisat-2) R 26684 26686 26686 0 -1 4202496 265404 0 0 0 9895 40 0 0 25 0 1 0 736841153 1206870016 227161 33554432000 4194304 4677096 140735067994272 18446744073709551615 4317711 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/26686/statm: 294646 227161 387 118 0 289605 0
[pid=26687] ppid=26644 vsize=717240 CPUtime=0.29 cores=1,3,5,7
/proc/26687/stat : 26687 (python) S 26644 26644 26612 0 -1 4202560 968 0 0 0 14 15 0 0 18 0 1 0 736841153 734453760 110145 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411782 0 0 16781312 514 9559805002768154885 0 0 17 1 0 0 0
/proc/26687/statm: 179310 110145 370 364 0 115222 0
[pid=26689] ppid=26687 vsize=149432 CPUtime=99.04 cores=1,3,5,7
/proc/26689/stat : 26689 (lingeling) R 26687 26689 26689 0 -1 4202496 69585 0 0 0 9891 13 0 0 25 0 1 0 736841155 153018368 36527 33554432000 4194304 4352236 140737367031216 18446744073709551615 4328368 0 0 16781312 17506 0 0 0 17 1 0 0 0
/proc/26689/statm: 37358 36527 129 39 0 36438 0
[pid=26690] ppid=26644 vsize=727496 CPUtime=0.31 cores=1,3,5,7
/proc/26690/stat : 26690 (python) S 26644 26644 26612 0 -1 4202560 977 0 0 0 15 16 0 0 15 0 1 0 736841155 744955904 110149 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411782 0 0 16781312 514 9559805002768154886 0 0 17 7 0 0 0
/proc/26690/statm: 181874 110149 370 364 0 117783 0
[pid=26692] ppid=26690 vsize=1187648 CPUtime=97.08 cores=1,3,5,7
/proc/26692/stat : 26692 (cryptominisat-2) R 26690 26692 26692 0 -1 4202496 266855 0 0 0 9670 38 0 0 25 0 1 0 736841156 1216151552 226976 33554432000 4194304 4677096 140736992496288 18446744073709551615 4336189 0 0 16781312 2 0 0 0 17 7 0 0 0
/proc/26692/statm: 296912 226976 387 118 0 291871 0
[pid=26644/tid=26682] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26682/stat : 26682 (python) S 26642 26644 26612 0 -1 4202560 36 932 0 0 0 0 0 0 18 0 5 0 736841151 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 5 0 0 0
[pid=26644/tid=26685] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26685/stat : 26685 (python) S 26642 26644 26612 0 -1 4202560 22 932 0 0 0 0 0 0 20 0 5 0 736841153 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
[pid=26644/tid=26688] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26688/stat : 26688 (python) S 26642 26644 26612 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 5 0 736841155 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
[pid=26644/tid=26691] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26691/stat : 26691 (python) S 26642 26644 26612 0 -1 4202560 28 932 0 0 0 0 0 0 23 0 5 0 736841156 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1178.51
Current children cumulated vsize (KiB) 6250488

[startup+942.301 s]
/proc/loadavg: 4.78 3.13 1.91 7/413 26696
/proc/meminfo: memFree=9547884/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=737740 CPUtime=782.48 cores=1,3,5,7
/proc/26644/stat : 26644 (python) S 26642 26644 26612 0 -1 4202496 750584 932 1 0 78128 120 0 0 23 0 5 0 736762859 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/26644/statm: 184435 111656 1873 364 0 120344 0
[pid=26681] ppid=26644 vsize=696728 CPUtime=0.5 cores=1,3,5,7
/proc/26681/stat : 26681 (python) S 26644 26644 26612 0 -1 4202560 959 0 0 0 24 26 0 0 18 0 1 0 736841150 713449472 110143 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411727 0 0 16781312 514 9559805002768154888 0 0 17 7 0 0 0
/proc/26681/statm: 174182 110143 376 364 0 110100 0
[pid=26683] ppid=26681 vsize=159124 CPUtime=159.36 cores=1,3,5,7
/proc/26683/stat : 26683 (lingeling) R 26681 26683 26683 0 -1 4202496 80543 0 0 0 15921 15 0 0 25 0 1 0 736841152 162942976 38950 33554432000 4194304 4352236 140735027334336 18446744073709551615 4319056 0 0 16781312 17506 0 0 0 17 3 0 0 0
/proc/26683/statm: 39781 38950 129 39 0 38861 0
[pid=26684] ppid=26644 vsize=706984 CPUtime=0.65 cores=1,3,5,7
/proc/26684/stat : 26684 (python) S 26644 26644 26612 0 -1 4202560 1255 0 0 0 36 29 0 0 15 0 1 0 736841152 723951616 110142 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411782 0 0 16781312 514 2211210246729203975 0 0 17 5 0 0 0
/proc/26684/statm: 176746 110142 371 364 0 112661 0
[pid=26686] ppid=26684 vsize=3742420 CPUtime=159.04 cores=1,3,5,7
/proc/26686/stat : 26686 (cryptominisat-2) R 26684 26686 26686 0 -1 4202496 4890387 0 0 0 15195 709 0 0 25 0 1 0 736841153 3832238080 867312 33554432000 4194304 4677096 140735067994272 18446744073709551615 4365684 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/26686/statm: 935605 867315 390 118 0 930564 0
[pid=26687] ppid=26644 vsize=717240 CPUtime=0.48 cores=1,3,5,7
/proc/26687/stat : 26687 (python) S 26644 26644 26612 0 -1 4202560 984 0 0 0 22 26 0 0 18 0 1 0 736841153 734453760 110145 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411782 0 0 16781312 514 9559805002768154885 0 0 17 1 0 0 0
/proc/26687/statm: 179310 110145 370 364 0 115222 0
[pid=26689] ppid=26687 vsize=168504 CPUtime=158.85 cores=1,3,5,7
/proc/26689/stat : 26689 (lingeling) R 26687 26689 26689 0 -1 4202496 82856 0 0 0 15871 14 0 0 25 0 1 0 736841155 172548096 41295 33554432000 4194304 4352236 140737367031216 18446744073709551615 4207615 0 0 16781312 17506 0 0 0 17 1 0 0 0
/proc/26689/statm: 42126 41295 129 39 0 41206 0
[pid=26690] ppid=26644 vsize=727496 CPUtime=0.75 cores=1,3,5,7
/proc/26690/stat : 26690 (python) S 26644 26644 26612 0 -1 4202560 1305 0 0 0 46 29 0 0 15 0 1 0 736841155 744955904 110150 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411782 0 0 16781312 514 17395980397757563142 0 0 17 7 0 0 0
/proc/26690/statm: 181874 110150 371 364 0 117783 0
[pid=26692] ppid=26690 vsize=3126340 CPUtime=155.58 cores=1,3,5,7
/proc/26692/stat : 26692 (cryptominisat-2) R 26690 26692 26692 0 -1 4202496 4004394 0 0 0 14984 574 0 0 25 0 1 0 736841156 3201372160 713791 33554432000 4194304 4677096 140736992496288 18446744073709551615 4395205 0 0 16781312 2 0 0 0 17 7 0 0 0
/proc/26692/statm: 781585 713791 390 118 0 776544 0
[pid=26644/tid=26682] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26682/stat : 26682 (python) S 26642 26644 26612 0 -1 4202560 36 932 0 0 0 0 0 0 18 0 5 0 736841151 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 5 0 0 0
[pid=26644/tid=26685] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26685/stat : 26685 (python) S 26642 26644 26612 0 -1 4202560 22 932 0 0 0 0 0 0 20 0 5 0 736841153 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
[pid=26644/tid=26688] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26688/stat : 26688 (python) S 26642 26644 26612 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 5 0 736841155 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
[pid=26644/tid=26691] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26691/stat : 26691 (python) S 26642 26644 26612 0 -1 4202560 28 932 0 0 0 0 0 0 23 0 5 0 736841156 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1417.69
Current children cumulated vsize (KiB) 10782576



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+996.001 s]
/proc/loadavg: 5.00 3.47 2.10 6/413 26698
/proc/meminfo: memFree=4621168/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=737740 CPUtime=782.48 cores=1,3,5,7
/proc/26644/stat : 26644 (python) S 26642 26644 26612 0 -1 4202496 750584 932 1 0 78128 120 0 0 23 0 5 0 736762859 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/26644/statm: 184435 111656 1873 364 0 120344 0
[pid=26681] ppid=26644 vsize=696728 CPUtime=0.66 cores=1,3,5,7
/proc/26681/stat : 26681 (python) S 26644 26644 26612 0 -1 4202560 971 0 0 0 31 35 0 0 18 0 1 0 736841150 713449472 110143 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411727 0 0 16781312 514 9559805002768154888 0 0 17 7 0 0 0
/proc/26681/statm: 174182 110143 376 364 0 110100 0
[pid=26683] ppid=26681 vsize=162364 CPUtime=213.08 cores=1,3,5,7
/proc/26683/stat : 26683 (lingeling) R 26681 26683 26683 0 -1 4202496 81392 0 0 0 21292 16 0 0 25 0 1 0 736841152 166260736 39761 33554432000 4194304 4352236 140735027334336 18446744073709551615 4207874 0 0 16781312 17506 0 0 0 17 3 0 0 0
/proc/26683/statm: 40591 39761 129 39 0 39671 0
[pid=26684] ppid=26644 vsize=706984 CPUtime=0.91 cores=1,3,5,7
/proc/26684/stat : 26684 (python) S 26644 26644 26612 0 -1 4202560 1299 0 0 0 51 40 0 0 15 0 1 0 736841152 723951616 110142 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411782 0 0 16781312 514 2211210246729203975 0 0 17 5 0 0 0
/proc/26684/statm: 176746 110142 371 364 0 112661 0
[pid=26686] ppid=26684 vsize=6198104 CPUtime=212.49 cores=1,3,5,7
/proc/26686/stat : 26686 (cryptominisat-2) R 26684 26686 26686 0 -1 4202496 9071495 0 0 0 19917 1332 0 0 25 0 1 0 736841153 6346858496 1473746 33554432000 4194304 4677096 140735067994272 18446744073709551615 256415089004 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/26686/statm: 1549526 1473746 390 118 0 1544485 0
[pid=26687] ppid=26644 vsize=717240 CPUtime=0.64 cores=1,3,5,7
/proc/26687/stat : 26687 (python) S 26644 26644 26612 0 -1 4202560 1000 0 0 0 29 35 0 0 18 0 1 0 736841153 734453760 110145 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411782 0 0 16781312 514 9559805002768154885 0 0 17 1 0 0 0
/proc/26687/statm: 179310 110145 370 364 0 115222 0
[pid=26689] ppid=26687 vsize=171708 CPUtime=212.41 cores=1,3,5,7
/proc/26689/stat : 26689 (lingeling) R 26687 26689 26689 0 -1 4202496 83705 0 0 0 21227 14 0 0 25 0 1 0 736841155 175828992 42097 33554432000 4194304 4352236 140737367031216 18446744073709551615 4209088 0 0 16781312 17506 0 0 0 17 1 0 0 0
/proc/26689/statm: 42927 42097 129 39 0 42007 0
[pid=26690] ppid=26644 vsize=727496 CPUtime=1.03 cores=1,3,5,7
/proc/26690/stat : 26690 (python) S 26644 26644 26612 0 -1 4202560 1350 0 0 0 63 40 0 0 15 0 1 0 736841155 744955904 110150 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411782 0 0 16781312 514 17395980397757563142 0 0 17 7 0 0 0
/proc/26690/statm: 181874 110150 371 364 0 117783 0
[pid=26692] ppid=26690 vsize=5775876 CPUtime=208.12 cores=1,3,5,7
/proc/26692/stat : 26692 (cryptominisat-2) R 26690 26692 26692 0 -1 4202496 8242565 0 0 0 19634 1178 0 0 25 0 1 0 736841156 5914497024 1366492 33554432000 4194304 4677096 140736992496288 18446744073709551615 256415088967 0 0 16781312 2 0 0 0 17 7 0 0 0
/proc/26692/statm: 1443969 1366492 390 118 0 1438928 0
[pid=26644/tid=26682] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26682/stat : 26682 (python) S 26642 26644 26612 0 -1 4202560 36 932 0 0 0 0 0 0 18 0 5 0 736841151 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 5 0 0 0
[pid=26644/tid=26685] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26685/stat : 26685 (python) S 26642 26644 26612 0 -1 4202560 22 932 0 0 0 0 0 0 20 0 5 0 736841153 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
[pid=26644/tid=26688] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26688/stat : 26688 (python) S 26642 26644 26612 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 5 0 736841155 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
[pid=26644/tid=26691] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26691/stat : 26691 (python) S 26642 26644 26612 0 -1 4202560 28 932 0 0 0 0 0 0 23 0 5 0 736841156 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1631.82
Current children cumulated vsize (KiB) 15894240

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+996.001 s]
/proc/loadavg: 5.00 3.47 2.10 6/413 26698
/proc/meminfo: memFree=4621168/32950928 swapFree=67111200/67111528
[pid=26644] ppid=26642 vsize=737740 CPUtime=782.48 cores=1,3,5,7
/proc/26644/stat : 26644 (python) S 26642 26644 26612 0 -1 4202496 750584 932 1 0 78128 120 0 0 23 0 5 0 736762859 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/26644/statm: 184435 111656 1873 364 0 120344 0
[pid=26681] ppid=26644 vsize=696728 CPUtime=0.66 cores=1,3,5,7
/proc/26681/stat : 26681 (python) S 26644 26644 26612 0 -1 4202560 971 0 0 0 31 35 0 0 18 0 1 0 736841150 713449472 110143 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411727 0 0 16781312 514 9559805002768154888 0 0 17 7 0 0 0
/proc/26681/statm: 174182 110143 376 364 0 110100 0
[pid=26683] ppid=26681 vsize=162364 CPUtime=213.08 cores=1,3,5,7
/proc/26683/stat : 26683 (lingeling) R 26681 26683 26683 0 -1 4202496 81392 0 0 0 21292 16 0 0 25 0 1 0 736841152 166260736 39761 33554432000 4194304 4352236 140735027334336 18446744073709551615 4207874 0 0 16781312 17506 0 0 0 17 3 0 0 0
/proc/26683/statm: 40591 39761 129 39 0 39671 0
[pid=26684] ppid=26644 vsize=706984 CPUtime=0.91 cores=1,3,5,7
/proc/26684/stat : 26684 (python) S 26644 26644 26612 0 -1 4202560 1299 0 0 0 51 40 0 0 15 0 1 0 736841152 723951616 110142 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411782 0 0 16781312 514 2211210246729203975 0 0 17 5 0 0 0
/proc/26684/statm: 176746 110142 371 364 0 112661 0
[pid=26686] ppid=26684 vsize=6198104 CPUtime=212.49 cores=1,3,5,7
/proc/26686/stat : 26686 (cryptominisat-2) R 26684 26686 26686 0 -1 4202496 9071495 0 0 0 19917 1332 0 0 25 0 1 0 736841153 6346858496 1473746 33554432000 4194304 4677096 140735067994272 18446744073709551615 256415089004 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/26686/statm: 1549526 1473746 390 118 0 1544485 0
[pid=26687] ppid=26644 vsize=717240 CPUtime=0.64 cores=1,3,5,7
/proc/26687/stat : 26687 (python) S 26644 26644 26612 0 -1 4202560 1000 0 0 0 29 35 0 0 18 0 1 0 736841153 734453760 110145 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411782 0 0 16781312 514 9559805002768154885 0 0 17 1 0 0 0
/proc/26687/statm: 179310 110145 370 364 0 115222 0
[pid=26689] ppid=26687 vsize=171708 CPUtime=212.41 cores=1,3,5,7
/proc/26689/stat : 26689 (lingeling) R 26687 26689 26689 0 -1 4202496 83705 0 0 0 21227 14 0 0 25 0 1 0 736841155 175828992 42097 33554432000 4194304 4352236 140737367031216 18446744073709551615 4209088 0 0 16781312 17506 0 0 0 17 1 0 0 0
/proc/26689/statm: 42927 42097 129 39 0 42007 0
[pid=26690] ppid=26644 vsize=727496 CPUtime=1.03 cores=1,3,5,7
/proc/26690/stat : 26690 (python) S 26644 26644 26612 0 -1 4202560 1350 0 0 0 63 40 0 0 15 0 1 0 736841155 744955904 110150 33554432000 4194304 5683644 140734000423296 18446744073709551615 256415411782 0 0 16781312 514 17395980397757563142 0 0 17 7 0 0 0
/proc/26690/statm: 181874 110150 371 364 0 117783 0
[pid=26692] ppid=26690 vsize=5775876 CPUtime=208.12 cores=1,3,5,7
/proc/26692/stat : 26692 (cryptominisat-2) R 26690 26692 26692 0 -1 4202496 8242565 0 0 0 19634 1178 0 0 25 0 1 0 736841156 5914497024 1366492 33554432000 4194304 4677096 140736992496288 18446744073709551615 256415088967 0 0 16781312 2 0 0 0 17 7 0 0 0
/proc/26692/statm: 1443969 1366492 390 118 0 1438928 0
[pid=26644/tid=26682] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26682/stat : 26682 (python) S 26642 26644 26612 0 -1 4202560 36 932 0 0 0 0 0 0 18 0 5 0 736841151 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 5 0 0 0
[pid=26644/tid=26685] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26685/stat : 26685 (python) S 26642 26644 26612 0 -1 4202560 22 932 0 0 0 0 0 0 20 0 5 0 736841153 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
[pid=26644/tid=26688] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26688/stat : 26688 (python) S 26642 26644 26612 0 -1 4202560 26 932 0 0 0 0 0 0 21 0 5 0 736841155 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
[pid=26644/tid=26691] ppid=26642 vsize=737740 CPUtime=0 cores=1,3,5,7
/proc/26644/task/26691/stat : 26691 (python) S 26642 26644 26612 0 -1 4202560 28 932 0 0 0 0 0 0 23 0 5 0 736841156 755445760 111656 33554432000 4194304 5683644 140734000423296 18446744073709551615 256427215281 0 0 16781312 2 18446604467096799040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1631.82
Current children cumulated vsize (KiB) 15894240

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 26644 and gives
#  childrusage.ru_utime.tv_sec=781
#  childrusage.ru_utime.tv_usec=288226
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=239811
# CPU time returned by wait4() is 782.528
# while last known CPU time is 1631.82
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 996.079
CPU time (s): 1631.82
CPU user time (s): 1603.72
CPU system time (s): 28.1
CPU usage (%): 163.824
Max. virtual memory (cumulated for all children) (KiB): 15894240

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 781.288
system time used= 1.23981
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 751518
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2926
involuntary context switches= 1207

runsolver used 3.45347 second user time and 8.71268 second system time

The end

Launcher Data 

Begin job on node115 at 2011-05-15 15:15:22
IDJOB=3378761
IDBENCH=82387
IDSOLVER=1860
FILE ID=node115/3378761-1305465319
RUNJOBID= node115-1305465319-26631
PBS_JOBID= 13325056
Free space on /tmp= 73100 MiB

SOLVER NAME= borg-sat 11.03.02-par-industrial (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.020-NOTKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378761-1305465319/watcher-3378761-1305465319 -o /tmp/evaluation-result-3378761-1305465319/solver-3378761-1305465319 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3378761-1305465319.cnf -seed 2051470122 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5d1253e2dcc0e3e55992672756beadbc
RANDOM SEED=2051470122

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      18316612 kB
Buffers:        267628 kB
Cached:        3283604 kB
SwapCached:          0 kB
Active:       11107128 kB
Inactive:      3208960 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      18316612 kB
SwapTotal:    67111528 kB
SwapFree:     67111200 kB
Dirty:          197408 kB
Writeback:           0 kB
AnonPages:    10764740 kB
Mapped:          33284 kB
Slab:           203216 kB
PageTables:      47844 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 95339712 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72948 MiB
End job on node115 at 2011-05-15 15:32:00