Trace number 3377861

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-industrial (fixed)? (MO) 1146.97 1146.86

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.020-NOTKNOWN.cnf
MD5SUM5d1253e2dcc0e3e55992672756beadbc
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2074.86
Satisfiable
(Un)Satisfiability was proved
Number of variables756832
Number of clauses4116966
Sum of the clauses size9765566
Maximum clause length1178
Minimum clause length1
Number of clauses of size 1846
Number of clauses of size 24082840
Number of clauses of size 3640
Number of clauses of size 4800
Number of clauses of size 5800
Number of clauses of size over 531040

Solver Data 

0.32/0.45	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.32/0.45	c INFO: solving HOME/instance-3377861-1305454273.cnf
0.32/0.45	c INFO: computing task features
19.01/19.18	c INFO: computed variable-clause graph statistics
19.61/19.71	c INFO: computed clause balance statistics
82.90/83.07	c INFO: computed variable balance statistics
128.41/128.58	c INFO: computed small-clause counts
132.01/132.13	c INFO: computed Horn-clause counts
855.54/855.67	c INFO: computed variable graph statistics
855.54/855.69	c INFO: collected features for HOME/instance-3377861-1305454273.cnf in 854.29 s
855.54/855.70	c INFO: running lingeling-276@0 for 97 with 1545 remaining (b = 0.21)
955.83/955.98	c INFO: running cryptominisat-2.9.0@0 for 1445 with 1445 remaining (b = 0.55)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3377861-1305454273/watcher-3377861-1305454273 -o /tmp/evaluation-result-3377861-1305454273/solver-3377861-1305454273 -C 2400 -W 1300 -M 7680 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3377861-1305454273.cnf -seed 1787504498 -budget 2400 

running on 2 cores: 0,2

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.16 4.06 4.01 5/335 19926
/proc/meminfo: memFree=20290000/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=15780 CPUtime=0 cores=0,2
/proc/19926/stat : 19926 (python) S 19924 19926 18433 0 -1 4202496 902 0 0 0 0 0 0 0 22 0 1 0 657030038 16158720 784 33554432000 4194304 5683644 140735729673504 18446744073709551615 212436015104 0 2147483391 16781312 2 18446744071699207358 0 0 17 0 0 0 0
/proc/19926/statm: 3945 784 346 364 0 432 0

[startup+0.0903309 s]
/proc/loadavg: 4.16 4.06 4.01 5/335 19926
/proc/meminfo: memFree=20290000/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=99380 CPUtime=0.06 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 2865 0 0 0 4 2 0 0 23 0 1 0 657030038 101765120 2565 33554432000 4194304 5683644 140735729673504 18446744073709551615 4562986 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/19926/statm: 24845 2565 818 364 0 1696 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 99380

[startup+0.100294 s]
/proc/loadavg: 4.16 4.06 4.01 5/335 19926
/proc/meminfo: memFree=20290000/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=104236 CPUtime=0.07 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 3086 0 0 0 5 2 0 0 23 0 1 0 657030038 106737664 2768 33554432000 4194304 5683644 140735729673504 18446744073709551615 5114052 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/19926/statm: 26059 2768 847 364 0 1807 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104236

[startup+0.306299 s]
/proc/loadavg: 4.16 4.06 4.01 5/335 19926
/proc/meminfo: memFree=20290000/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=198920 CPUtime=0.24 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 7030 934 1 0 18 6 0 0 17 0 1 0 657030038 203694080 5053 33554432000 4194304 5683644 140735729673504 18446744073709551615 4620076 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 49730 5053 1466 364 0 3757 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 198920

[startup+0.700174 s]
/proc/loadavg: 4.16 4.06 4.01 5/335 19926
/proc/meminfo: memFree=20290000/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=370720 CPUtime=0.61 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 32280 934 1 0 49 12 0 0 18 0 1 0 657030038 379617280 29949 33554432000 4194304 5683644 140735729673504 18446744073709551615 4961433 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/19926/statm: 92680 29949 1761 364 0 28604 0
Current children cumulated CPU time (s) 0.61
Current children cumulated vsize (KiB) 370720

[startup+1.50106 s]
/proc/loadavg: 4.16 4.06 4.01 5/336 19931
/proc/meminfo: memFree=20125072/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=421536 CPUtime=1.41 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 44985 934 1 0 127 14 0 0 19 0 1 0 657030038 431652864 42654 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988704948946 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 105384 42655 1761 364 0 41308 0
Current children cumulated CPU time (s) 1.41
Current children cumulated vsize (KiB) 421536

[startup+3.10075 s]
/proc/loadavg: 4.16 4.06 4.01 5/336 19931
/proc/meminfo: memFree=19984376/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=521120 CPUtime=3 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 69426 934 1 0 282 18 0 0 22 0 1 0 657030038 533626880 67095 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988704976995 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 130280 67096 1761 364 0 66204 0
Current children cumulated CPU time (s) 3
Current children cumulated vsize (KiB) 521120

[startup+6.30019 s]
/proc/loadavg: 4.23 4.08 4.02 5/336 19931
/proc/meminfo: memFree=19783876/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=720060 CPUtime=6.2 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 117994 934 1 0 595 25 0 0 25 0 1 0 657030038 737341440 115663 33554432000 4194304 5683644 140735729673504 18446744073709551615 4602993 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 180015 115664 1761 364 0 115939 0
Current children cumulated CPU time (s) 6.2
Current children cumulated vsize (KiB) 720060

[startup+12.701 s]
/proc/loadavg: 4.21 4.07 4.01 5/336 19931
/proc/meminfo: memFree=19760080/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=1085860 CPUtime=12.61 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 314600 934 1 0 1209 52 0 0 25 0 1 0 657030038 1111920640 200829 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988534071152 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 271465 200829 1814 364 0 207389 0
Current children cumulated CPU time (s) 12.61
Current children cumulated vsize (KiB) 1085860

[startup+25.5005 s]
/proc/loadavg: 4.23 4.09 4.02 5/339 19935
/proc/meminfo: memFree=19453092/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=989372 CPUtime=25.4 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 539311 934 1 0 2462 78 0 0 25 0 1 0 657030038 1013116928 184759 33554432000 4194304 5683644 140735729673504 18446744073709551615 4418259 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184759 1826 364 0 183267 0
Current children cumulated CPU time (s) 25.4
Current children cumulated vsize (KiB) 989372

[startup+51.1007 s]
/proc/loadavg: 4.22 4.09 4.02 5/339 19935
/proc/meminfo: memFree=19407192/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=989372 CPUtime=51 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 539311 934 1 0 5022 78 0 0 25 0 1 0 657030038 1013116928 184759 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988534235437 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184759 1826 364 0 183267 0
Current children cumulated CPU time (s) 51
Current children cumulated vsize (KiB) 989372

[startup+102.312 s]
/proc/loadavg: 4.09 4.08 4.02 5/339 19937
/proc/meminfo: memFree=19291688/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=989372 CPUtime=102.22 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 539312 934 1 0 10144 78 0 0 25 0 1 0 657030038 1013116928 184760 33554432000 4194304 5683644 140735729673504 18446744073709551615 4455460 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184760 1827 364 0 183267 0
Current children cumulated CPU time (s) 102.22
Current children cumulated vsize (KiB) 989372

[startup+162.301 s]
/proc/loadavg: 4.03 4.06 4.01 5/339 19938
/proc/meminfo: memFree=19290184/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=989372 CPUtime=162.21 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 747955 934 1 0 16119 102 0 0 25 0 1 0 657030038 1013116928 184766 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988534085772 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184766 1833 364 0 183267 0
Current children cumulated CPU time (s) 162.21
Current children cumulated vsize (KiB) 989372

[startup+222.3 s]
/proc/loadavg: 4.01 4.05 4.00 5/339 19940
/proc/meminfo: memFree=19290708/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=989372 CPUtime=222.21 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 747955 934 1 0 22119 102 0 0 25 0 1 0 657030038 1013116928 184766 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988534085772 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184766 1833 364 0 183267 0
Current children cumulated CPU time (s) 222.21
Current children cumulated vsize (KiB) 989372

[startup+282.301 s]
/proc/loadavg: 4.00 4.04 4.00 5/339 19941
/proc/meminfo: memFree=19285596/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=989372 CPUtime=282.22 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 747955 934 1 0 28120 102 0 0 25 0 1 0 657030038 1013116928 184766 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988534085760 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184766 1833 364 0 183267 0
Current children cumulated CPU time (s) 282.22
Current children cumulated vsize (KiB) 989372

[startup+342.301 s]
/proc/loadavg: 4.00 4.03 4.00 5/339 19943
/proc/meminfo: memFree=19285780/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=989372 CPUtime=342.22 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 747955 934 1 0 34120 102 0 0 25 0 1 0 657030038 1013116928 184766 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988534085772 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184766 1833 364 0 183267 0
Current children cumulated CPU time (s) 342.22
Current children cumulated vsize (KiB) 989372

[startup+402.3 s]
/proc/loadavg: 4.07 4.04 4.00 5/339 19945
/proc/meminfo: memFree=19279904/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=989372 CPUtime=402.22 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 747955 934 1 0 40120 102 0 0 25 0 1 0 657030038 1013116928 184766 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988534085760 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184766 1833 364 0 183267 0
Current children cumulated CPU time (s) 402.22
Current children cumulated vsize (KiB) 989372

[startup+462.301 s]
/proc/loadavg: 4.08 4.04 4.00 5/339 19946
/proc/meminfo: memFree=19279960/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=989372 CPUtime=462.22 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 747955 934 1 0 46120 102 0 0 25 0 1 0 657030038 1013116928 184766 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988534085760 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184766 1833 364 0 183267 0
Current children cumulated CPU time (s) 462.22
Current children cumulated vsize (KiB) 989372

[startup+522.3 s]
/proc/loadavg: 4.03 4.03 4.00 5/342 19951
/proc/meminfo: memFree=19363092/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=989372 CPUtime=522.22 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 747955 934 1 0 52120 102 0 0 25 0 1 0 657030038 1013116928 184766 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988534085772 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184766 1833 364 0 183267 0
Current children cumulated CPU time (s) 522.22
Current children cumulated vsize (KiB) 989372

[startup+582.301 s]
/proc/loadavg: 4.13 4.07 4.01 5/346 19956
/proc/meminfo: memFree=18997456/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=989372 CPUtime=582.23 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 747955 934 1 0 58121 102 0 0 25 0 1 0 657030038 1013116928 184766 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988566764401 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184766 1833 364 0 183267 0
Current children cumulated CPU time (s) 582.23
Current children cumulated vsize (KiB) 989372

[startup+642.301 s]

################
# More data... #
################

[pid=19926] ppid=19924 vsize=989372 CPUtime=822.24 cores=0,2
/proc/19926/stat : 19926 (python) R 19924 19926 18433 0 -1 4202496 747955 934 1 0 82121 103 0 0 25 0 1 0 657030038 1013116928 184766 33554432000 4194304 5683644 140735729673504 18446744073709551615 46988534085760 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/19926/statm: 247343 184766 1833 364 0 183267 0
Current children cumulated CPU time (s) 822.24
Current children cumulated vsize (KiB) 989372

[startup+882.3 s]
/proc/loadavg: 3.20 3.70 3.88 4/344 19992
/proc/meminfo: memFree=19028556/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=706980 CPUtime=855.64 cores=0,2
/proc/19926/stat : 19926 (python) S 19924 19926 18433 0 -1 4202496 748238 934 1 0 85459 105 0 0 25 0 2 0 657030038 723947520 111639 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/19926/statm: 176745 111639 1867 364 0 112663 0
[pid=19990] ppid=19926 vsize=696736 CPUtime=0.06 cores=0,2
/proc/19990/stat : 19990 (python) S 19926 19926 18433 0 -1 4202560 895 0 0 0 3 3 0 0 18 0 1 0 657115608 713457664 110144 33554432000 4194304 5683644 140735729673504 18446744073709551615 212436037135 0 0 16781312 514 6618180389909266692 0 0 17 0 0 0 0
/proc/19990/statm: 174184 110144 376 364 0 110102 0
[pid=19992] ppid=19990 vsize=149364 CPUtime=26.5 cores=0,2
/proc/19992/stat : 19992 (lingeling) R 19990 19992 19992 0 -1 4202496 69261 0 0 0 2638 12 0 0 25 0 1 0 657115609 152948736 36509 33554432000 4194304 4352236 140736362808272 18446744073709551615 4207627 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/19992/statm: 37341 36509 127 39 0 36421 0
[pid=19926/tid=19991] ppid=19924 vsize=706980 CPUtime=0 cores=0,2
/proc/19926/task/19991/stat : 19991 (python) S 19924 19926 18433 0 -1 4202560 29 934 0 0 0 0 0 0 25 0 2 0 657115609 723947520 111639 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 2 18446604445215389888 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 882.2
Current children cumulated vsize (KiB) 1553080

[startup+942.301 s]
/proc/loadavg: 3.19 3.60 3.83 4/348 19998
/proc/meminfo: memFree=17530004/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=706980 CPUtime=855.64 cores=0,2
/proc/19926/stat : 19926 (python) S 19924 19926 18433 0 -1 4202496 748238 934 1 0 85459 105 0 0 25 0 2 0 657030038 723947520 111639 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/19926/statm: 176745 111639 1867 364 0 112663 0
[pid=19990] ppid=19926 vsize=696736 CPUtime=0.22 cores=0,2
/proc/19990/stat : 19990 (python) S 19926 19926 18433 0 -1 4202560 924 0 0 0 11 11 0 0 18 0 1 0 657115608 713457664 110144 33554432000 4194304 5683644 140735729673504 18446744073709551615 212436037135 0 0 16781312 514 6618180389909266692 0 0 17 0 0 0 0
/proc/19990/statm: 174184 110144 376 364 0 110102 0
[pid=19992] ppid=19990 vsize=148252 CPUtime=86.36 cores=0,2
/proc/19992/stat : 19992 (lingeling) R 19990 19992 19992 0 -1 4202496 69263 0 0 0 8624 12 0 0 25 0 1 0 657115609 151810048 36233 33554432000 4194304 4352236 140736362808272 18446744073709551615 4235610 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/19992/statm: 37063 36233 129 39 0 36143 0
[pid=19926/tid=19991] ppid=19924 vsize=706980 CPUtime=0 cores=0,2
/proc/19926/task/19991/stat : 19991 (python) S 19924 19926 18433 0 -1 4202560 29 934 0 0 0 0 0 0 25 0 2 0 657115609 723947520 111639 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 2 18446604445215389888 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 942.22
Current children cumulated vsize (KiB) 1551968

[startup+1002.3 s]
/proc/loadavg: 3.07 3.49 3.78 4/352 20004
/proc/meminfo: memFree=16449856/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=717236 CPUtime=855.66 cores=0,2
/proc/19926/stat : 19926 (python) S 19924 19926 18433 0 -1 4202496 749058 934 1 0 85460 106 0 0 19 0 3 0 657030038 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/19926/statm: 179309 111646 1870 364 0 115224 0
[pid=19990] ppid=19926 vsize=706980 CPUtime=0.26 cores=0,2
/proc/19990/stat : 19990 (python) S 19926 19926 18433 0 -1 4202560 1165 0 0 0 13 13 0 0 19 0 2 0 657115608 723947520 110187 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/19990/statm: 176745 110187 414 364 0 112663 0
[pid=19992] ppid=19990 vsize=148664 CPUtime=99.99 cores=0,2
/proc/19992/stat : 19992 (lingeling) T 19990 19992 19992 0 -1 4202496 69394 0 0 0 9987 12 0 0 25 0 1 0 657115609 152231936 36336 33554432000 4194304 4352236 140736362808272 18446744073709551615 4331241 0 0 16781312 17506 18446744071562691360 0 0 17 2 0 0 0
/proc/19992/statm: 37166 36336 129 39 0 36246 0
[pid=19990/tid=19999] ppid=19926 vsize=706980 CPUtime=0 cores=0,2
/proc/19990/task/19999/stat : 19999 (python) S 19926 19926 18433 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657125636 723947520 110187 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 514 18446604445288706112 0 0 -1 0 0 0 0
[pid=20000] ppid=19926 vsize=706992 CPUtime=0.13 cores=0,2
/proc/20000/stat : 20000 (python) S 19926 19926 18433 0 -1 4202560 933 0 0 0 7 6 0 0 15 0 1 0 657125636 723959808 110142 33554432000 4194304 5683644 140735729673504 18446744073709551615 212436037190 0 0 16781312 514 6618180389909266690 0 0 17 0 0 0 0
/proc/20000/statm: 176748 110142 370 364 0 112663 0
[pid=20002] ppid=20000 vsize=1206668 CPUtime=46.14 cores=0,2
/proc/20002/stat : 20002 (cryptominisat-2) R 20000 20002 20002 0 -1 4202496 241084 0 0 0 4583 31 0 0 25 0 1 0 657125638 1235628032 217322 33554432000 4194304 4677096 140737011165712 18446744073709551615 4319643 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/20002/statm: 301667 217322 387 118 0 296626 0
[pid=19926/tid=19991] ppid=19924 vsize=717236 CPUtime=0 cores=0,2
/proc/19926/task/19991/stat : 19991 (python) S 19924 19926 18433 0 -1 4202560 29 934 0 0 0 0 0 0 25 0 3 0 657115609 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 2 18446604445215389888 0 0 -1 0 0 0 0
[pid=19926/tid=20001] ppid=19924 vsize=717236 CPUtime=0 cores=0,2
/proc/19926/task/20001/stat : 20001 (python) S 19924 19926 18433 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 3 0 657125638 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 2 18446604445215389888 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1002.18
Current children cumulated vsize (KiB) 3486540

[startup+1062.3 s]
/proc/loadavg: 2.80 3.35 3.71 3/339 20023
/proc/meminfo: memFree=19251452/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=717236 CPUtime=855.66 cores=0,2
/proc/19926/stat : 19926 (python) S 19924 19926 18433 0 -1 4202496 749058 934 1 0 85460 106 0 0 19 0 3 0 657030038 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/19926/statm: 179309 111646 1870 364 0 115224 0
[pid=19990] ppid=19926 vsize=706980 CPUtime=0.26 cores=0,2
/proc/19990/stat : 19990 (python) S 19926 19926 18433 0 -1 4202560 1165 0 0 0 13 13 0 0 19 0 2 0 657115608 723947520 110187 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/19990/statm: 176745 110187 414 364 0 112663 0
[pid=19992] ppid=19990 vsize=148664 CPUtime=99.99 cores=0,2
/proc/19992/stat : 19992 (lingeling) T 19990 19992 19992 0 -1 4202496 69394 0 0 0 9987 12 0 0 25 0 1 0 657115609 152231936 36336 33554432000 4194304 4352236 140736362808272 18446744073709551615 4331241 0 0 16781312 17506 18446744071562691360 0 0 17 2 0 0 0
/proc/19992/statm: 37166 36336 129 39 0 36246 0
[pid=19990/tid=19999] ppid=19926 vsize=706980 CPUtime=0 cores=0,2
/proc/19990/task/19999/stat : 19999 (python) S 19926 19926 18433 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657125636 723947520 110187 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 514 18446604445288706112 0 0 -1 0 0 0 0
[pid=20000] ppid=19926 vsize=706992 CPUtime=0.31 cores=0,2
/proc/20000/stat : 20000 (python) S 19926 19926 18433 0 -1 4202560 1201 0 0 0 16 15 0 0 15 0 1 0 657125636 723959808 110143 33554432000 4194304 5683644 140735729673504 18446744073709551615 212436037190 0 0 16781312 514 14364556471234953474 0 0 17 0 0 0 0
/proc/20000/statm: 176748 110143 371 364 0 112663 0
[pid=20002] ppid=20000 vsize=1455628 CPUtime=106.04 cores=0,2
/proc/20002/stat : 20002 (cryptominisat-2) R 20000 20002 20002 0 -1 4202496 340271 0 0 0 10559 45 0 0 18 0 1 0 657125638 1490563072 290885 33554432000 4194304 4677096 140737011165712 18446744073709551615 4293298 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/20002/statm: 363907 290885 390 118 0 358866 0
[pid=19926/tid=19991] ppid=19924 vsize=717236 CPUtime=0 cores=0,2
/proc/19926/task/19991/stat : 19991 (python) S 19924 19926 18433 0 -1 4202560 29 934 0 0 0 0 0 0 25 0 3 0 657115609 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 2 18446604445215389888 0 0 -1 0 0 0 0
[pid=19926/tid=20001] ppid=19924 vsize=717236 CPUtime=0 cores=0,2
/proc/19926/task/20001/stat : 20001 (python) S 19924 19926 18433 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 3 0 657125638 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 2 18446604445215389888 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1062.26
Current children cumulated vsize (KiB) 3735500

[startup+1122.3 s]
/proc/loadavg: 2.35 3.12 3.60 3/343 20029
/proc/meminfo: memFree=16151584/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=717236 CPUtime=855.66 cores=0,2
/proc/19926/stat : 19926 (python) S 19924 19926 18433 0 -1 4202496 749058 934 1 0 85460 106 0 0 19 0 3 0 657030038 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/19926/statm: 179309 111646 1870 364 0 115224 0
[pid=19990] ppid=19926 vsize=706980 CPUtime=0.26 cores=0,2
/proc/19990/stat : 19990 (python) S 19926 19926 18433 0 -1 4202560 1165 0 0 0 13 13 0 0 19 0 2 0 657115608 723947520 110187 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/19990/statm: 176745 110187 414 364 0 112663 0
[pid=19992] ppid=19990 vsize=148664 CPUtime=99.99 cores=0,2
/proc/19992/stat : 19992 (lingeling) T 19990 19992 19992 0 -1 4202496 69394 0 0 0 9987 12 0 0 25 0 1 0 657115609 152231936 36336 33554432000 4194304 4352236 140736362808272 18446744073709551615 4331241 0 0 16781312 17506 18446744071562691360 0 0 17 2 0 0 0
/proc/19992/statm: 37166 36336 129 39 0 36246 0
[pid=19990/tid=19999] ppid=19926 vsize=706980 CPUtime=0 cores=0,2
/proc/19990/task/19999/stat : 19999 (python) S 19926 19926 18433 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657125636 723947520 110187 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 514 18446604445288706112 0 0 -1 0 0 0 0
[pid=20000] ppid=19926 vsize=706992 CPUtime=0.59 cores=0,2
/proc/20000/stat : 20000 (python) S 19926 19926 18433 0 -1 4202560 1263 0 0 0 35 24 0 0 15 0 1 0 657125636 723959808 110143 33554432000 4194304 5683644 140735729673504 18446744073709551615 212436037190 0 0 16781312 514 14364556471234953474 0 0 17 0 0 0 0
/proc/20000/statm: 176748 110143 371 364 0 112663 0
[pid=20002] ppid=20000 vsize=4388936 CPUtime=165.89 cores=0,2
/proc/20002/stat : 20002 (cryptominisat-2) R 20000 20002 20002 0 -1 4202496 5958543 0 0 0 15858 731 0 0 25 0 1 0 657125638 4494270464 1023698 33554432000 4194304 4677096 140737011165712 18446744073709551615 4548319 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/20002/statm: 1097234 1023698 390 118 0 1092193 0
[pid=19926/tid=19991] ppid=19924 vsize=717236 CPUtime=0 cores=0,2
/proc/19926/task/19991/stat : 19991 (python) S 19924 19926 18433 0 -1 4202560 29 934 0 0 0 0 0 0 25 0 3 0 657115609 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 2 18446604445215389888 0 0 -1 0 0 0 0
[pid=19926/tid=20001] ppid=19924 vsize=717236 CPUtime=0 cores=0,2
/proc/19926/task/20001/stat : 20001 (python) S 19924 19926 18433 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 3 0 657125638 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 2 18446604445215389888 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1122.39
Current children cumulated vsize (KiB) 6668808



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1146.8 s]
/proc/loadavg: 2.23 3.02 3.56 3/343 20030
/proc/meminfo: memFree=14776156/32951124 swapFree=57331800/67111528
[pid=19926] ppid=19924 vsize=717236 CPUtime=855.66 cores=0,2
/proc/19926/stat : 19926 (python) S 19924 19926 18433 0 -1 4202496 749058 934 1 0 85460 106 0 0 19 0 3 0 657030038 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/19926/statm: 179309 111646 1870 364 0 115224 0
[pid=19990] ppid=19926 vsize=706980 CPUtime=0.26 cores=0,2
/proc/19990/stat : 19990 (python) S 19926 19926 18433 0 -1 4202560 1165 0 0 0 13 13 0 0 19 0 2 0 657115608 723947520 110187 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/19990/statm: 176745 110187 414 364 0 112663 0
[pid=19992] ppid=19990 vsize=148664 CPUtime=99.99 cores=0,2
/proc/19992/stat : 19992 (lingeling) T 19990 19992 19992 0 -1 4202496 69394 0 0 0 9987 12 0 0 25 0 1 0 657115609 152231936 36336 33554432000 4194304 4352236 140736362808272 18446744073709551615 4331241 0 0 16781312 17506 18446744071562691360 0 0 17 2 0 0 0
/proc/19992/statm: 37166 36336 129 39 0 36246 0
[pid=19990/tid=19999] ppid=19926 vsize=706980 CPUtime=0 cores=0,2
/proc/19990/task/19999/stat : 19999 (python) S 19926 19926 18433 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657125636 723947520 110187 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 514 18446604445288706112 0 0 -1 0 0 0 0
[pid=20000] ppid=19926 vsize=706992 CPUtime=0.71 cores=0,2
/proc/20000/stat : 20000 (python) S 19926 19926 18433 0 -1 4202560 1288 0 0 0 43 28 0 0 15 0 1 0 657125636 723959808 110143 33554432000 4194304 5683644 140735729673504 18446744073709551615 212436037190 0 0 16781312 514 14364556471234953474 0 0 17 0 0 0 0
/proc/20000/statm: 176748 110143 371 364 0 112663 0
[pid=20002] ppid=20000 vsize=5586268 CPUtime=190.35 cores=0,2
/proc/20002/stat : 20002 (cryptominisat-2) R 20000 20002 20002 0 -1 4202496 8237791 0 0 0 18031 1004 0 0 25 0 1 0 657125638 5720338432 1324733 33554432000 4194304 4677096 140737011165712 18446744073709551615 4548991 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/20002/statm: 1396567 1324733 390 118 0 1391526 0
[pid=19926/tid=19991] ppid=19924 vsize=717236 CPUtime=0 cores=0,2
/proc/19926/task/19991/stat : 19991 (python) S 19924 19926 18433 0 -1 4202560 29 934 0 0 0 0 0 0 25 0 3 0 657115609 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 2 18446604445215389888 0 0 -1 0 0 0 0
[pid=19926/tid=20001] ppid=19924 vsize=717236 CPUtime=0 cores=0,2
/proc/19926/task/20001/stat : 20001 (python) S 19924 19926 18433 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 3 0 657125638 734449664 111646 33554432000 4194304 5683644 140735729673504 18446744073709551615 212447840689 0 0 16781312 2 18446604445215389888 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1146.97
Current children cumulated vsize (KiB) 7866140

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 19926 and gives
#  childrusage.ru_utime.tv_sec=854
#  childrusage.ru_utime.tv_usec=605080
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=91834
# CPU time returned by wait4() is 855.697
# while last known CPU time is 1146.97
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1146.86
CPU time (s): 1146.97
CPU user time (s): 1135.34
CPU system time (s): 11.63
CPU usage (%): 100.01
Max. virtual memory (cumulated for all children) (KiB): 7866140

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 854.605
system time used= 1.09183
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 749992
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 979
involuntary context switches= 1979

runsolver used 3.86641 second user time and 7.76882 second system time

The end

Launcher Data 

Begin job on node102 at 2011-05-15 12:11:15
IDJOB=3377861
IDBENCH=82387
IDSOLVER=1858
FILE ID=node102/3377861-1305454273
RUNJOBID= node102-1305452895-19554
PBS_JOBID= 13324852
Free space on /tmp= 73180 MiB

SOLVER NAME= borg-sat 11.03.02-seq-industrial (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.020-NOTKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3377861-1305454273/watcher-3377861-1305454273 -o /tmp/evaluation-result-3377861-1305454273/solver-3377861-1305454273 -C 2400 -W 1300 -M 7680  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3377861-1305454273.cnf -seed 1787504498 -budget 2400

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= 5d1253e2dcc0e3e55992672756beadbc
RANDOM SEED=1787504498

node102.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      20290404 kB
Buffers:         19628 kB
Cached:         846240 kB
SwapCached:      89884 kB
Active:        3507080 kB
Inactive:      8954192 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      20290404 kB
SwapTotal:    67111528 kB
SwapFree:     57331800 kB
Dirty:          459920 kB
Writeback:           0 kB
AnonPages:    11516072 kB
Mapped:          18644 kB
Slab:            72240 kB
PageTables:      60776 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 61953784 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73492 MiB
End job on node102 at 2011-05-15 12:30:26