Trace number 3321573

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-industrial? (MO) 4169.33 2304.99

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-city-sequential-35nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.040-NOTKNOWN.cnf
MD5SUMce0d180356398dfbcfd2bbbb1cb3f9e5
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark363.759
Satisfiable
(Un)Satisfiability was proved
Number of variables1260306
Number of clauses6771840
Sum of the clauses size16095280
Maximum clause length982
Minimum clause length1
Number of clauses of size 1720
Number of clauses of size 26714640
Number of clauses of size 3320
Number of clauses of size 41280
Number of clauses of size 53200
Number of clauses of size over 551680

Solver Data 

0.34/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.34/0.67	c INFO: solving HOME/instance-3321573-1304347119.cnf
0.34/0.67	c INFO: computing task features
35.92/36.21	c INFO: computed variable-clause graph statistics
36.71/37.02	c INFO: computed clause balance statistics
136.31/136.66	c INFO: computed variable balance statistics
205.01/205.31	c INFO: computed small-clause counts
213.82/214.13	c INFO: computed Horn-clause counts
2031.97/2032.20	c INFO: computed variable graph statistics
2031.97/2032.22	c INFO: collected features for HOME/instance-3321573-1304347119.cnf in 2029.61 s
2031.97/2032.26	c INFO: running cryptominisat-2.9.0@0 for 1067 with 37968 remaining (b = 0.61)
2032.07/2032.30	c INFO: running lingeling-276@0 for 388 with 37968 remaining (b = 0.30)
2032.07/2032.35	c INFO: running cryptominisat-2.9.0@0 for 1940 with 37968 remaining (b = 0.71)
2032.07/2032.39	c INFO: running glucose@0 for 5821 with 37968 remaining (b = 0.87)
2032.17/2032.44	c INFO: running lingeling-276@0 for 97 with 37968 remaining (b = 0.10)
2032.17/2032.48	c INFO: running cryptominisat-2.9.0@0 for 1940 with 37968 remaining (b = 0.57)
2032.27/2032.53	c INFO: running glucose@0 for 5821 with 37968 remaining (b = 0.84)
2032.27/2032.58	c INFO: running glucose@0 for 5821 with 37968 remaining (b = 0.83)
2816.83/2130.67	c INFO: running glucose@0 for 5821 with 37870 remaining (b = 0.82)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321573-1304347119/watcher-3321573-1304347119 -o /tmp/evaluation-result-3321573-1304347119/solver-3321573-1304347119 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3321573-1304347119.cnf -seed 568608205 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.90 7.80 7.92 2/199 25146
/proc/meminfo: memFree=13937688/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=15380 CPUtime=0 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 808 0 0 0 0 0 0 0 21 0 1 0 624943007 15749120 693 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367546512 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25146/statm: 3845 693 332 364 0 332 0

[startup+0.093522 s]
/proc/loadavg: 6.90 7.80 7.92 2/199 25146
/proc/meminfo: memFree=13937688/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=84600 CPUtime=0.05 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 2408 0 0 0 3 2 0 0 18 0 1 0 624943007 86630400 2135 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367544832 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25146/statm: 21150 2135 707 364 0 1367 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 84600

[startup+0.100524 s]
/proc/loadavg: 6.90 7.80 7.92 2/199 25146
/proc/meminfo: memFree=13937688/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=87700 CPUtime=0.05 cores=0-7
/proc/25146/stat : 25146 (python) S 25144 25146 25120 0 -1 4202496 2637 0 0 0 3 2 0 0 17 0 1 0 624943007 89804800 2353 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367544832 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/25146/statm: 21925 2353 717 364 0 1621 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 87700

[startup+0.300449 s]
/proc/loadavg: 6.90 7.80 7.92 2/199 25146
/proc/meminfo: memFree=13937688/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=135748 CPUtime=0.17 cores=0-7
/proc/25146/stat : 25146 (python) S 25144 25146 25120 0 -1 4202496 5964 933 1 0 13 4 0 0 18 0 1 0 624943007 139005952 4150 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367542789 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/25146/statm: 33937 4150 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135748

[startup+0.700355 s]
/proc/loadavg: 6.90 7.80 7.92 2/199 25146
/proc/meminfo: memFree=13937688/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=412064 CPUtime=0.41 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 19494 933 1 0 29 12 0 0 17 0 1 0 624943007 421953536 17175 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367546624 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25146/statm: 103016 17221 1758 364 0 38940 0
Current children cumulated CPU time (s) 0.41
Current children cumulated vsize (KiB) 412064

[startup+1.50023 s]
/proc/loadavg: 6.90 7.80 7.92 2/200 25151
/proc/meminfo: memFree=13768608/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=462072 CPUtime=1.21 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 54954 933 1 0 102 19 0 0 18 0 1 0 624943007 473161728 52635 33554432000 4194304 5683644 140735193407040 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25146/statm: 115518 52636 1761 364 0 51442 0
Current children cumulated CPU time (s) 1.21
Current children cumulated vsize (KiB) 462072

[startup+3.10092 s]
/proc/loadavg: 6.90 7.80 7.92 2/200 25151
/proc/meminfo: memFree=13696316/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=568644 CPUtime=2.81 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 80710 933 1 0 258 23 0 0 20 0 1 0 624943007 582291456 78391 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367207621 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25146/statm: 142161 78391 1761 364 0 78085 0
Current children cumulated CPU time (s) 2.81
Current children cumulated vsize (KiB) 568644

[startup+6.30033 s]
/proc/loadavg: 6.42 7.68 7.88 2/200 25153
/proc/meminfo: memFree=13485412/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=775664 CPUtime=6.01 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 131949 933 1 0 571 30 0 0 25 0 1 0 624943007 794279936 129630 33554432000 4194304 5683644 140735193407040 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25146/statm: 193916 129631 1761 364 0 129840 0
Current children cumulated CPU time (s) 6.01
Current children cumulated vsize (KiB) 775664

[startup+12.7011 s]
/proc/loadavg: 5.99 7.57 7.85 2/200 25153
/proc/meminfo: memFree=13063400/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1182100 CPUtime=12.41 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 234159 933 1 0 1197 44 0 0 25 0 1 0 624943007 1210470400 231840 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367207621 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25146/statm: 295525 231841 1761 364 0 231449 0
Current children cumulated CPU time (s) 12.41
Current children cumulated vsize (KiB) 1182100

[startup+25.5008 s]
/proc/loadavg: 4.88 7.25 7.73 2/200 25154
/proc/meminfo: memFree=12627360/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1595472 CPUtime=25.21 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 895887 933 1 0 2407 114 0 0 25 0 1 0 624943007 1633763328 325340 33554432000 4194304 5683644 140735193407040 18446744073709551615 47081861039738 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25146/statm: 398868 325341 1814 364 0 334792 0
Current children cumulated CPU time (s) 25.21
Current children cumulated vsize (KiB) 1595472

[startup+51.101 s]
/proc/loadavg: 3.56 6.75 7.55 2/200 25155
/proc/meminfo: memFree=12804044/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1393684 CPUtime=50.81 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 1147349 933 1 0 4940 141 0 0 25 0 1 0 624943007 1427132416 285806 33554432000 4194304 5683644 140735193407040 18446744073709551615 47081863342496 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25146/statm: 348421 285806 1826 364 0 284345 0
Current children cumulated CPU time (s) 50.81
Current children cumulated vsize (KiB) 1393684

[startup+102.307 s]
/proc/loadavg: 2.11 5.86 7.21 2/200 25156
/proc/meminfo: memFree=12804716/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1393684 CPUtime=102.02 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 1147349 933 1 0 10061 141 0 0 25 0 1 0 624943007 1427132416 285806 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367206323 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25146/statm: 348421 285806 1826 364 0 284345 0
Current children cumulated CPU time (s) 102.02
Current children cumulated vsize (KiB) 1393684

[startup+162.3 s]
/proc/loadavg: 1.41 4.97 6.82 2/200 25158
/proc/meminfo: memFree=12804580/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1393684 CPUtime=162.01 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 1147350 933 1 0 16060 141 0 0 25 0 1 0 624943007 1427132416 285807 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367205834 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25146/statm: 348421 285807 1827 364 0 284345 0
Current children cumulated CPU time (s) 162.01
Current children cumulated vsize (KiB) 1393684

[startup+222.3 s]
/proc/loadavg: 1.15 4.24 6.45 2/200 25160
/proc/meminfo: memFree=12804292/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1393684 CPUtime=222.01 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 1742571 933 1 0 21993 208 0 0 25 0 1 0 624943007 1427132416 285813 33554432000 4194304 5683644 140735193407040 18446744073709551615 47081860626572 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25146/statm: 348421 285813 1833 364 0 284345 0
Current children cumulated CPU time (s) 222.01
Current children cumulated vsize (KiB) 1393684

[startup+282.3 s]
/proc/loadavg: 1.08 3.66 6.11 2/200 25161
/proc/meminfo: memFree=12804496/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1393684 CPUtime=282.01 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 1742571 933 1 0 27993 208 0 0 25 0 1 0 624943007 1427132416 285813 33554432000 4194304 5683644 140735193407040 18446744073709551615 47081860626572 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25146/statm: 348421 285813 1833 364 0 284345 0
Current children cumulated CPU time (s) 282.01
Current children cumulated vsize (KiB) 1393684

[startup+342.3 s]
/proc/loadavg: 1.03 3.18 5.79 2/200 25163
/proc/meminfo: memFree=12804600/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1393684 CPUtime=342.02 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 1742571 933 1 0 33994 208 0 0 25 0 1 0 624943007 1427132416 285813 33554432000 4194304 5683644 140735193407040 18446744073709551615 47081860626572 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25146/statm: 348421 285813 1833 364 0 284345 0
Current children cumulated CPU time (s) 342.02
Current children cumulated vsize (KiB) 1393684

[startup+402.301 s]
/proc/loadavg: 1.01 2.78 5.49 2/200 25164
/proc/meminfo: memFree=12804436/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1393684 CPUtime=402.02 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 1742571 933 1 0 39994 208 0 0 25 0 1 0 624943007 1427132416 285813 33554432000 4194304 5683644 140735193407040 18446744073709551615 47081860626572 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25146/statm: 348421 285813 1833 364 0 284345 0
Current children cumulated CPU time (s) 402.02
Current children cumulated vsize (KiB) 1393684

[startup+462.301 s]
/proc/loadavg: 1.00 2.45 5.20 2/200 25166
/proc/meminfo: memFree=12804652/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1393684 CPUtime=462.02 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 1742571 933 1 0 45994 208 0 0 25 0 1 0 624943007 1427132416 285813 33554432000 4194304 5683644 140735193407040 18446744073709551615 47081860626572 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25146/statm: 348421 285813 1833 364 0 284345 0
Current children cumulated CPU time (s) 462.02
Current children cumulated vsize (KiB) 1393684

[startup+522.301 s]
/proc/loadavg: 1.00 2.19 4.94 2/200 25168
/proc/meminfo: memFree=12804492/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1393684 CPUtime=522.02 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 1742571 933 1 0 51994 208 0 0 25 0 1 0 624943007 1427132416 285813 33554432000 4194304 5683644 140735193407040 18446744073709551615 47081860626572 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25146/statm: 348421 285813 1833 364 0 284345 0
Current children cumulated CPU time (s) 522.02
Current children cumulated vsize (KiB) 1393684

[startup+582.301 s]
/proc/loadavg: 1.00 1.97 4.69 2/200 25169
/proc/meminfo: memFree=12804572/32950928 swapFree=67111352/67111528
[pid=25146] ppid=25144 vsize=1393684 CPUtime=582.02 cores=0-7
/proc/25146/stat : 25146 (python) R 25144 25146 25120 0 -1 4202496 1742571 933 1 0 57994 208 0 0 25 0 1 0 624943007 1427132416 285813 33554432000 4194304 5683644 140735193407040 18446744073709551615 47081860626572 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25146/statm: 348421 285813 1833 364 0 284345 0
Current children cumulated CPU time (s) 582.02
Current children cumulated vsize (KiB) 1393684

[startup+642.301 s]

################
# More data... #
################



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+2303.4 s]
/proc/loadavg: 8.61 5.42 3.42 8/228 25254
/proc/meminfo: memFree=130424/32950928 swapFree=63945796/67111528
[pid=25146] ppid=25144 vsize=1069236 CPUtime=2032.42 cores=0-7
/proc/25146/stat : 25146 (python) S 25144 25146 25120 0 -1 4202496 1749223 933 1 0 203015 227 0 0 19 0 10 0 624943007 1094897664 164022 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379373515 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/25146/statm: 267309 164022 579 364 0 203203 0
[pid=25218] ppid=25146 vsize=911408 CPUtime=0.7 cores=0-7
/proc/25218/stat : 25218 (python) S 25146 25146 25120 0 -1 4202560 1216 0 0 0 41 29 0 0 15 0 1 0 625146233 933281792 163303 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367566863 0 0 16781312 514 5628839831531389188 0 0 17 4 0 0 0
/proc/25218/statm: 227852 163303 234 364 0 163770 0
[pid=25220] ppid=25218 vsize=7855504 CPUtime=262.98 cores=0-7
/proc/25220/stat : 25220 (cryptominisat-2) R 25218 25220 25220 0 -1 4202496 11257924 0 0 0 24774 1524 0 0 19 0 1 0 625146235 8044036096 1862709 33554432000 4194304 4677096 140736143225488 18446744073709551615 4365684 0 0 16781312 2 0 0 0 17 1 0 0 673
/proc/25220/statm: 1963876 1862711 258 118 0 1958835 0
[pid=25221] ppid=25146 vsize=921664 CPUtime=0.43 cores=0-7
/proc/25221/stat : 25221 (python) S 25146 25146 25120 0 -1 4202560 977 0 0 0 23 20 0 0 18 0 1 0 625146238 943783936 163148 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367566918 0 0 16781312 514 1624251353108807939 0 0 17 6 0 0 23
/proc/25221/statm: 230416 163148 234 364 0 166331 0
[pid=25223] ppid=25221 vsize=296224 CPUtime=271.73 cores=0-7
/proc/25223/stat : 25223 (lingeling) R 25221 25223 25223 0 -1 4202496 206089 0 0 0 27141 32 0 0 25 0 1 0 625146240 303333376 73226 33554432000 4194304 4352236 140735607045600 18446744073709551615 4207615 0 0 16781312 17506 0 0 0 17 2 0 0 0
/proc/25223/statm: 74056 73226 129 39 0 73136 0
[pid=25224] ppid=25146 vsize=931920 CPUtime=0.54 cores=0-7
/proc/25224/stat : 25224 (python) S 25146 25146 25120 0 -1 4202560 1219 0 0 0 29 25 0 0 15 0 1 0 625146242 954286080 163347 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367566918 0 0 16781312 514 7968877652368916740 0 0 17 3 0 0 70
/proc/25224/statm: 232980 163347 235 364 0 168892 0
[pid=25226] ppid=25224 vsize=5768136 CPUtime=264.83 cores=0-7
/proc/25226/stat : 25226 (cryptominisat-2) R 25224 25226 25226 0 -1 4202496 7604677 0 0 0 25342 1141 0 0 20 0 1 0 625146244 5906571264 1274229 33554432000 4194304 4677096 140733844677904 18446744073709551615 4563764 0 0 16781312 2 0 0 0 17 7 0 0 598
/proc/25226/statm: 1442034 1274229 258 118 0 1436993 0
[pid=25227] ppid=25146 vsize=942176 CPUtime=0.41 cores=0-7
/proc/25227/stat : 25227 (python) S 25146 25146 25120 0 -1 4202560 998 0 0 0 20 21 0 0 18 0 1 0 625146247 964788224 163204 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367566918 0 0 16781312 514 1624251353108807938 0 0 17 6 0 0 0
/proc/25227/statm: 235544 163204 234 364 0 171453 0
[pid=25229] ppid=25227 vsize=716800 CPUtime=271.39 cores=0-7
/proc/25229/stat : 25229 (glucose_static) R 25227 25229 25229 0 -1 4202496 168974 0 0 0 27105 34 0 0 25 0 1 0 625146248 734003200 168507 33554432000 134512640 135217611 4287660000 18446744073709551615 134527815 0 0 16781312 8388611 0 0 0 17 4 0 0 0
/proc/25229/statm: 179200 168507 78 173 0 179025 0
[pid=25230] ppid=25146 vsize=1017968 CPUtime=0.17 cores=0-7
/proc/25230/stat : 25230 (python) S 25146 25146 25120 0 -1 4202560 1127 0 0 0 8 9 0 0 19 0 2 0 625146251 1042399232 163241 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379373515 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/25230/statm: 254492 163241 244 364 0 190398 0
[pid=25232] ppid=25230 vsize=272852 CPUtime=97.33 cores=0-7
/proc/25232/stat : 25232 (lingeling) T 25230 25232 25232 0 -1 4202496 175506 0 0 0 9705 28 0 0 25 0 1 0 625146253 279400448 67383 33554432000 4194304 4352236 140736200121088 18446744073709551615 4278708 0 0 16781312 17506 18446744071562691360 0 0 17 7 0 0 0
/proc/25232/statm: 68213 67383 129 39 0 67293 0
[pid=25230/tid=25245] ppid=25146 vsize=1017968 CPUtime=0 cores=0-7
/proc/25230/task/25245/stat : 25245 (python) S 25146 25146 25120 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 625156071 1042399232 163241 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379370417 0 0 16781312 514 18446604447755625600 0 0 -1 7 0 0 0
[pid=25233] ppid=25146 vsize=1028224 CPUtime=0.51 cores=0-7
/proc/25233/stat : 25233 (python) S 25146 25146 25120 0 -1 4202560 1240 0 0 0 29 22 0 0 15 0 1 0 625146256 1052901376 163359 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367566918 0 0 16781312 514 4356225491124846852 0 0 17 6 0 0 37
/proc/25233/statm: 257056 163359 235 364 0 192959 0
[pid=25235] ppid=25233 vsize=4845660 CPUtime=263.96 cores=0-7
/proc/25235/stat : 25235 (cryptominisat-2) R 25233 25235 25235 0 -1 4202496 5483082 0 0 0 25527 869 0 0 19 0 1 0 625146258 4961955840 1083513 33554432000 4194304 4677096 140736259157984 18446744073709551615 4260568 0 0 16781312 2 0 0 0 17 6 0 0 606
/proc/25235/statm: 1211415 1083523 258 118 0 1206374 0
[pid=25236] ppid=25146 vsize=1038480 CPUtime=0.43 cores=0-7
/proc/25236/stat : 25236 (python) S 25146 25146 25120 0 -1 4202560 998 0 0 0 23 20 0 0 18 0 1 0 625146261 1063403520 163240 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367566918 0 0 16781312 514 1624251353108807940 0 0 17 6 0 0 0
/proc/25236/statm: 259620 163240 234 364 0 195520 0
[pid=25238] ppid=25236 vsize=716800 CPUtime=269.44 cores=0-7
/proc/25238/stat : 25238 (glucose_static) R 25236 25238 25238 0 -1 4202496 168978 0 0 0 26909 35 0 0 25 0 1 0 625146264 734003200 168506 33554432000 134512640 135217611 4290473568 18446744073709551615 134537887 0 0 16781312 8388611 0 0 0 17 5 0 0 18
/proc/25238/statm: 179200 168506 78 173 0 179025 0
[pid=25239] ppid=25146 vsize=1048736 CPUtime=0.42 cores=0-7
/proc/25239/stat : 25239 (python) S 25146 25146 25120 0 -1 4202560 1038 0 0 0 21 21 0 0 18 0 1 0 625146265 1073905664 163296 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367566918 0 0 16781312 514 1624251353108807939 0 0 17 7 0 0 0
/proc/25239/statm: 262184 163296 234 364 0 198081 0
[pid=25241] ppid=25239 vsize=716796 CPUtime=270.67 cores=0-7
/proc/25241/stat : 25241 (glucose_static) R 25239 25241 25241 0 -1 4202496 168988 0 0 0 27035 32 0 0 25 0 1 0 625146267 733999104 168506 33554432000 134512640 135217611 4294949408 18446744073709551615 134527905 0 0 16781312 8388611 0 0 0 17 3 0 0 0
/proc/25241/statm: 179199 168506 78 173 0 179024 0
[pid=25246] ppid=25146 vsize=1058992 CPUtime=0.27 cores=0-7
/proc/25246/stat : 25246 (python) S 25146 25146 25120 0 -1 4202560 981 0 0 0 14 13 0 0 18 0 1 0 625156074 1084407808 163639 33554432000 4194304 5683644 140735193407040 18446744073709551615 251367566918 0 0 16781312 514 1624251353108807940 0 0 17 0 0 0 49
/proc/25246/statm: 264748 163639 234 364 0 200642 0
[pid=25248] ppid=25246 vsize=619260 CPUtime=159.32 cores=0-7
/proc/25248/stat : 25248 (glucose_static) D 25246 25248 25248 0 -1 4202496 144533 0 69 0 15899 33 0 0 18 0 1 0 625156076 634122240 138973 33554432000 134512640 135217611 4287248528 18446744073709551615 134533127 0 0 16781312 8388611 18446744071562234166 0 0 17 0 0 0 1385
/proc/25248/statm: 154815 138973 78 173 0 154640 0
[pid=25146/tid=25219] ppid=25144 vsize=1069236 CPUtime=0 cores=0-7
/proc/25146/task/25219/stat : 25219 (python) S 25144 25146 25120 0 -1 4202560 25 933 0 0 0 0 0 0 25 0 10 0 625146235 1094897664 164022 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379370417 0 0 16781312 2 18446604447755626496 0 0 -1 4 0 0 0
[pid=25146/tid=25222] ppid=25144 vsize=1069236 CPUtime=0 cores=0-7
/proc/25146/task/25222/stat : 25222 (python) S 25144 25146 25120 0 -1 4202560 22 933 0 0 0 0 0 0 20 0 10 0 625146239 1094897664 164022 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379370417 0 0 16781312 2 18446604447755626496 0 0 -1 0 0 0 0
[pid=25146/tid=25225] ppid=25144 vsize=1069236 CPUtime=0 cores=0-7
/proc/25146/task/25225/stat : 25225 (python) S 25144 25146 25120 0 -1 4202560 24 933 0 0 0 0 0 0 21 0 10 0 625146244 1094897664 164022 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379370417 0 0 16781312 2 18446604447755626496 0 0 -1 4 0 0 0
[pid=25146/tid=25228] ppid=25144 vsize=1069236 CPUtime=0 cores=0-7
/proc/25146/task/25228/stat : 25228 (python) S 25144 25146 25120 0 -1 4202560 18 933 0 0 0 0 0 0 19 0 10 0 625146248 1094897664 164022 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379370417 0 0 16781312 2 18446604447755626496 0 0 -1 5 0 0 0
[pid=25146/tid=25231] ppid=25144 vsize=1069236 CPUtime=0 cores=0-7
/proc/25146/task/25231/stat : 25231 (python) S 25144 25146 25120 0 -1 4202560 23 933 0 0 0 0 0 0 19 0 10 0 625146253 1094897664 164022 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379370417 0 0 16781312 2 18446604447755626496 0 0 -1 0 0 0 0
[pid=25146/tid=25234] ppid=25144 vsize=1069236 CPUtime=0 cores=0-7
/proc/25146/task/25234/stat : 25234 (python) S 25144 25146 25120 0 -1 4202560 24 933 0 0 0 0 0 0 21 0 10 0 625146257 1094897664 164022 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379370417 0 0 16781312 2 18446604447755626496 0 0 -1 1 0 0 0
[pid=25146/tid=25237] ppid=25144 vsize=1069236 CPUtime=0 cores=0-7
/proc/25146/task/25237/stat : 25237 (python) S 25144 25146 25120 0 -1 4202560 24 933 0 0 0 0 0 0 22 0 10 0 625146262 1094897664 164022 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379370417 0 0 16781312 2 18446604447755626496 0 0 -1 1 0 0 0
[pid=25146/tid=25240] ppid=25144 vsize=1069236 CPUtime=0 cores=0-7
/proc/25146/task/25240/stat : 25240 (python) S 25144 25146 25120 0 -1 4202560 29 933 0 0 0 0 0 0 19 0 10 0 625146267 1094897664 164022 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379370417 0 0 16781312 2 18446604447755626496 0 0 -1 1 0 0 0
[pid=25146/tid=25247] ppid=25144 vsize=1069236 CPUtime=0 cores=0-7
/proc/25146/task/25247/stat : 25247 (python) S 25144 25146 25120 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 10 0 625156076 1094897664 164022 33554432000 4194304 5683644 140735193407040 18446744073709551615 251379370417 0 0 16781312 2 18446604447755626496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4167.95
Current children cumulated vsize (KiB) 31776836

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+2304.9 s]
/proc/loadavg: 8.61 5.42 3.42 8/228 25254
/proc/meminfo: memFree=130424/32950928 swapFree=63945796/67111528
[pid=25146] ppid=25144 vsize=0 CPUtime=2032.47 cores=0-7
/proc/25146/stat : 25146 (python) Z 25144 25146 25120 0 -1 4203548 1749230 933 1 0 203015 232 0 0 18 0 2 0 624943007 0 0 33554432000 0 0 0 0 0 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/25146/statm: 0 0 0 0 0 0 0
[pid=25218] ppid=25146 vsize=0 CPUtime=0.71 cores=0-7
/proc/25218/stat : 25218 (python) Z 25146 25146 25120 0 -1 4203612 1216 0 0 0 41 30 0 0 16 0 1 0 625146233 0 0 33554432000 0 0 0 0 0 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/25218/statm: 0 0 0 0 0 0 0
[pid=25220] ppid=1 vsize=0 CPUtime=263.22 cores=0-7
/proc/25220/stat : 25220 (cryptominisat-2) R 1 25220 25220 0 -1 4203524 11264643 0 0 0 24776 1546 0 0 19 0 1 0 625146235 0 0 33554432000 0 0 0 0 0 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 673
/proc/25220/statm: 0 0 0 0 0 0 0
[pid=25221] ppid=25146 vsize=0 CPUtime=0.44 cores=0-7
/proc/25221/stat : 25221 (python) Z 25146 25146 25120 0 -1 4203612 977 0 0 0 23 21 0 0 15 0 1 0 625146238 0 0 33554432000 0 0 0 0 0 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 23
/proc/25221/statm: 0 0 0 0 0 0 0
[pid=25223] ppid=1 vsize=0 CPUtime=271.79 cores=0-7
/proc/25223/stat : 25223 (lingeling) Z 1 25223 25223 0 -1 4203548 206090 0 0 0 27146 33 0 0 25 0 1 0 625146240 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 2 0 0 0
/proc/25223/statm: 0 0 0 0 0 0 0
[pid=25224] ppid=25146 vsize=0 CPUtime=0.56 cores=0-7
/proc/25224/stat : 25224 (python) Z 25146 25146 25120 0 -1 4203612 1219 0 0 0 29 27 0 0 15 0 1 0 625146242 0 0 33554432000 0 0 0 0 0 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 70
/proc/25224/statm: 0 0 0 0 0 0 0
[pid=25226] ppid=1 vsize=0 CPUtime=265.08 cores=0-7
/proc/25226/stat : 25226 (cryptominisat-2) R 1 25226 25226 0 -1 4203524 7606909 0 0 0 25347 1161 0 0 20 0 1 0 625146244 0 0 33554432000 0 0 0 0 0 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 598
/proc/25226/statm: 0 0 0 0 0 0 0
[pid=25227] ppid=25146 vsize=0 CPUtime=0.42 cores=0-7
/proc/25227/stat : 25227 (python) Z 25146 25146 25120 0 -1 4203612 998 0 0 0 20 22 0 0 18 0 1 0 625146247 0 0 33554432000 0 0 0 0 0 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/25227/statm: 0 0 0 0 0 0 0
[pid=25229] ppid=1 vsize=0 CPUtime=271.52 cores=0-7
/proc/25229/stat : 25229 (glucose_static) Z 1 25229 25229 0 -1 4203548 168974 0 0 0 27111 41 0 0 25 0 1 0 625146248 0 0 33554432000 0 0 0 0 0 0 0 16781312 8388611 18446744073709551615 0 0 17 6 0 0 0
/proc/25229/statm: 0 0 0 0 0 0 0
[pid=25230] ppid=25146 vsize=0 CPUtime=0.18 cores=0-7
/proc/25230/stat : 25230 (python) Z 25146 25146 25120 0 -1 4203612 1127 0 0 0 8 10 0 0 18 0 1 0 625146251 0 0 33554432000 0 0 0 0 0 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/25230/statm: 0 0 0 0 0 0 0
[pid=25232] ppid=1 vsize=272852 CPUtime=97.33 cores=0-7
/proc/25232/stat : 25232 (lingeling) T 1 25232 25232 0 -1 4202496 175506 0 0 0 9705 28 0 0 25 0 1 0 625146253 279400448 67383 33554432000 4194304 4352236 140736200121088 18446744073709551615 4278708 0 0 16781312 17506 18446744071562691360 0 0 17 7 0 0 0
/proc/25232/statm: 68213 67383 129 39 0 67293 0
[pid=25233] ppid=25146 vsize=0 CPUtime=0.52 cores=0-7
/proc/25233/stat : 25233 (python) Z 25146 25146 25120 0 -1 4203612 1240 0 0 0 29 23 0 0 15 0 1 0 625146256 0 0 33554432000 0 0 0 0 0 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 37
/proc/25233/statm: 0 0 0 0 0 0 0
[pid=25235] ppid=1 vsize=0 CPUtime=264.22 cores=0-7
/proc/25235/stat : 25235 (cryptominisat-2) R 1 25235 25235 0 -1 4203524 5485453 0 0 0 25533 889 0 0 19 0 1 0 625146258 0 0 33554432000 0 0 0 0 0 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 606
/proc/25235/statm: 0 0 0 0 0 0 0
[pid=25236] ppid=25146 vsize=0 CPUtime=0.44 cores=0-7
/proc/25236/stat : 25236 (python) Z 25146 25146 25120 0 -1 4203612 998 0 0 0 23 21 0 0 15 0 1 0 625146261 0 0 33554432000 0 0 0 0 0 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/25236/statm: 0 0 0 0 0 0 0
[pid=25238] ppid=1 vsize=0 CPUtime=269.58 cores=0-7
/proc/25238/stat : 25238 (glucose_static) Z 1 25238 25238 0 -1 4203548 168978 0 0 0 26916 42 0 0 25 0 1 0 625146264 0 0 33554432000 0 0 0 0 0 0 0 16781312 8388611 18446744073709551615 0 0 17 5 0 0 18
/proc/25238/statm: 0 0 0 0 0 0 0
[pid=25239] ppid=25146 vsize=0 CPUtime=0.44 cores=0-7
/proc/25239/stat : 25239 (python) Z 25146 25146 25120 0 -1 4203612 1038 0 0 0 21 23 0 0 15 0 1 0 625146265 0 0 33554432000 0 0 0 0 0 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/25239/statm: 0 0 0 0 0 0 0
[pid=25241] ppid=1 vsize=0 CPUtime=270.81 cores=0-7
/proc/25241/stat : 25241 (glucose_static) Z 1 25241 25241 0 -1 4203548 168988 0 0 0 27042 39 0 0 25 0 1 0 625146267 0 0 33554432000 0 0 0 0 0 0 0 16781312 8388611 18446744073709551615 0 0 17 2 0 0 0
/proc/25241/statm: 0 0 0 0 0 0 0
[pid=25246] ppid=25146 vsize=0 CPUtime=0.28 cores=0-7
/proc/25246/stat : 25246 (python) Z 25146 25146 25120 0 -1 4203612 981 0 0 0 14 14 0 0 15 0 1 0 625156074 0 0 33554432000 0 0 0 0 0 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 49
/proc/25246/statm: 0 0 0 0 0 0 0
[pid=25248] ppid=1 vsize=619260 CPUtime=159.32 cores=0-7
/proc/25248/stat : 25248 (glucose_static) D 1 25248 25248 0 -1 4202496 144544 0 72 0 15899 33 0 0 18 0 1 0 625156076 634122240 138980 33554432000 134512640 135217611 4287248528 18446744073709551615 134533127 256 0 16781312 8388611 18446744071562234166 0 0 17 0 0 0 1394
/proc/25248/statm: 154815 138980 78 173 0 154640 0
[pid=25146/tid=25247] ppid=25144 vsize=0 CPUtime=0.04 cores=0-7
/proc/25146/task/25247/stat : 25247 (python) D 25144 25146 25120 0 -1 4203588 28 933 0 0 0 4 0 0 18 0 2 0 625156076 0 0 33554432000 0 0 0 0 0 0 0 16781312 2 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4169.33
Current children cumulated vsize (KiB) 892112

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25146 and gives
#  childrusage.ru_utime.tv_sec=2030
#  childrusage.ru_utime.tv_usec=156369
#  childrusage.ru_stime.tv_sec=2
#  childrusage.ru_stime.tv_usec=331645
# CPU time returned by wait4() is 2032.49
# while last known CPU time is 4169.33
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 2304.99
CPU time (s): 4169.33
CPU user time (s): 4126.98
CPU system time (s): 42.35
CPU usage (%): 180.882
Max. virtual memory (cumulated for all children) (KiB): 31844396

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2030.16
system time used= 2.33164
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1750163
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2992
involuntary context switches= 25643

runsolver used 4.07338 second user time and 10.4594 second system time

The end

Launcher Data 

Begin job on node113 at 2011-05-02 16:38:43
IDJOB=3321573
IDBENCH=82374
IDSOLVER=1710
FILE ID=node113/3321573-1304347119
RUNJOBID= node113-1304347119-25134
PBS_JOBID= 13178654
Free space on /tmp= 73444 MiB

SOLVER NAME= borg-sat 11.03.02-industrial
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-city-sequential-35nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.040-NOTKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321573-1304347119/watcher-3321573-1304347119 -o /tmp/evaluation-result-3321573-1304347119/solver-3321573-1304347119 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3321573-1304347119.cnf -seed 568608205 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= ce0d180356398dfbcfd2bbbb1cb3f9e5
RANDOM SEED=568608205

node113.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      13937720 kB
Buffers:        830004 kB
Cached:        2927076 kB
SwapCached:         12 kB
Active:       15480020 kB
Inactive:      2991772 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      13937720 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:          165796 kB
Writeback:           0 kB
AnonPages:    14715208 kB
Mapped:          19388 kB
Slab:           439924 kB
PageTables:      38320 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 17261748 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73412 MiB
End job on node113 at 2011-05-02 17:17:10