Trace number 3728947

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 1712.2 1708.57

General information on the benchmark

Namenormalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/
prestwich/robin/normalized-robin18.opb
MD5SUMf17e31cb6762bd0e8d3e659d7bd1050c
Bench CategoryDEC-SMALLINT (no optimisation, small integers)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
SatisfiableNO
(Un)Satisfiability was provedYES
Best value of the objective function
Optimality of the best value was proved
Number of variables23409
Total number of constraints1386
Number of constraints which are clauses612
Number of constraints which are cardinality constraints (but not clauses)774
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint153
Maximum length of a constraint289
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 2
Number of bits of the biggest number in a constraint 2
Biggest sum of numbers in a constraint 291
Number of bits of the biggest sum of numbers9
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.35/0.59	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.35/0.59	c INFO: solving HOME/instance-3728947-1338713247.opb
0.70/0.94	c INFO: parsing took 0.35 s
1.31/1.57	c INFO: features took 0.62 CPU seconds
1.40/1.60	c INFO: running clasp-1.3.7@0 for 45 with 1798 remaining (b = 0.17)
48.54/48.75	c INFO: running scip-2.0.1-spx@0 for 45 with 1751 remaining (b = 0.03)
96.38/96.54	c INFO: running scip-2.0.1-clp@0 for 45 with 1704 remaining (b = 0.02)
145.83/145.93	c INFO: running scip-2.0.1-clp@49 for 45 with 1654 remaining (b = 0.01)
190.01/190.08	c INFO: running clasp-1.3.7@47 for 45 with 1610 remaining (b = 0.01)
237.99/237.92	c INFO: wrote minion input file to HOME/tmp0c6cRT.minion
237.99/237.92	c INFO: running minion-0.12@0 for 45 with 1565 remaining (b = 0.08)
285.10/285.08	c INFO: running clasp-1.3.7@92 for 136 with 1516 remaining (b = 0.02)
422.85/422.51	c INFO: running scip-2.0.1-spx@47 for 45 with 1378 remaining (b = 0.01)
468.02/467.66	c INFO: running scip-2.0.1-spx@92 for 410 with 1333 remaining (b = 0.06)
879.29/878.26	c INFO: running bsolo_pb10-l1@0 for 91 with 923 remaining (b = 0.01)
975.83/974.62	c INFO: running scip-2.0.1-spx@502 for 547 with 828 remaining (b = 0.05)
1525.57/1523.41	c INFO: running wbo1.4a@0 for 45 with 280 remaining (b = 0.00)
1574.12/1572.00	c INFO: running sat4j-pb-v20101225@0 for 45 with 232 remaining (b = 0.00)
1620.76/1617.27	c INFO: running scip-2.0.1-clp@93 for 45 with 185 remaining (b = 0.00)
1666.01/1662.42	c INFO: running clasp-1.3.7@229 for 45 with 140 remaining (b = 0.00)
1711.19/1707.57	c INFO: running pbct-0.1.2-linear@0 for 95 with 95 remaining (b = 0.00)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3728947-1338713247/watcher-3728947-1338713247 -o /tmp/evaluation-result-3728947-1338713247/solver-3728947-1338713247 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3728947-1338713247.opb -seed 808678085 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.42 1.31 1.63 3/179 3829
/proc/meminfo: memFree=26951184/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=15308 CPUtime=0 cores=1,3,5,7
/proc/3829/stat : 3829 (python) R 3827 3829 3621 0 -1 4202496 660 0 0 0 0 0 0 0 20 0 1 0 337449611 15675392 554 33554432000 4194304 5683644 140735745319744 140735745305656 4540219 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 3827 556 316 364 0 269 0

[startup+0.0669421 s]
/proc/loadavg: 1.42 1.31 1.63 3/179 3829
/proc/meminfo: memFree=26951184/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=39100 CPUtime=0.05 cores=1,3,5,7
/proc/3829/stat : 3829 (python) D 3827 3829 3621 0 -1 4202496 1787 0 0 0 4 1 0 0 20 0 1 0 337449611 40038400 1470 33554432000 4194304 5683644 140735745319744 140735745273464 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 9775 1470 464 364 0 1030 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 39100

[startup+0.10026 s]
/proc/loadavg: 1.42 1.31 1.63 3/179 3829
/proc/meminfo: memFree=26951184/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=92388 CPUtime=0.07 cores=1,3,5,7
/proc/3829/stat : 3829 (python) R 3827 3829 3621 0 -1 4202496 3190 507 1 0 5 2 0 0 20 0 1 0 337449611 94605312 2540 33554432000 4194304 5683644 140735745319744 140735745272216 213419263401 0 0 16781312 2 0 0 0 17 3 0 0 0 0 0
/proc/3829/statm: 23097 2540 816 364 0 1619 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 92388

[startup+0.300236 s]
/proc/loadavg: 1.42 1.31 1.63 3/179 3829
/proc/meminfo: memFree=26951184/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=166944 CPUtime=0.22 cores=1,3,5,7
/proc/3829/stat : 3829 (python) R 3827 3829 3621 0 -1 4202496 6560 1012 1 0 16 6 0 0 20 0 1 0 337449611 170950656 4351 33554432000 4194304 5683644 140735745319744 140735745223976 4297424 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 41736 4351 1227 364 0 3289 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 166944

[startup+0.700217 s]
/proc/loadavg: 1.42 1.31 1.63 3/179 3829
/proc/meminfo: memFree=26951184/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=286484 CPUtime=0.5 cores=1,3,5,7
/proc/3829/stat : 3829 (python) R 3827 3829 3621 0 -1 4202496 11432 1012 1 0 40 10 0 0 20 0 1 0 337449611 293359616 8332 33554432000 4194304 5683644 140735745319744 140735745312696 213419265867 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 71621 8332 1811 364 0 6850 0
Current children cumulated CPU time (s) 0.5
Current children cumulated vsize (KiB) 286484

[startup+1.50018 s]
/proc/loadavg: 1.42 1.31 1.63 3/180 3834
/proc/meminfo: memFree=26914820/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=294696 CPUtime=1.31 cores=1,3,5,7
/proc/3829/stat : 3829 (python) R 3827 3829 3621 0 -1 4202496 15908 1012 1 0 120 11 0 0 20 0 1 0 337449611 301768704 10427 33554432000 4194304 5683644 140735745319744 140735745312696 213427185433 0 0 16781312 33554434 0 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 73674 10427 1847 364 0 8903 0
Current children cumulated CPU time (s) 1.31
Current children cumulated vsize (KiB) 294696

[startup+3.10035 s]
/proc/loadavg: 1.42 1.31 1.63 3/183 3837
/proc/meminfo: memFree=26899924/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=370500 CPUtime=1.4 cores=1,3,5,7
/proc/3829/stat : 3829 (python) S 3827 3829 3621 0 -1 4202496 16197 1012 1 0 129 11 0 0 20 0 2 0 337449611 379392000 10485 33554432000 4194304 5683644 140735745319744 140735745314576 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 92625 10485 1894 364 0 27848 0
[pid=3835] ppid=3829 vsize=294720 CPUtime=0 cores=1,3,5,7
/proc/3835/stat : 3835 (python) S 3829 3829 3621 0 -1 4202560 742 0 0 0 0 0 0 0 20 0 1 0 337449771 301793280 8925 33554432000 4194304 5683644 140735745319744 140735745310832 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/3835/statm: 73680 8925 340 364 0 8903 0
[pid=3837] ppid=3835 vsize=18560 CPUtime=1.48 cores=1,3,5,7
/proc/3837/stat : 3837 (clasp-1.3.7-x86) R 3835 3837 3837 0 -1 4202496 4987 0 0 0 147 1 0 0 20 0 1 0 337449771 19005440 4376 33554432000 134512640 136139019 4292475168 4292473680 134627181 0 0 16781312 16386 0 0 0 17 1 0 0 0 0 0
/proc/3837/statm: 4640 4376 181 398 0 4240 0
[pid=3829/tid=3836] ppid=3827 vsize=370500 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3836/stat : 3836 (python) S 3827 3829 3621 0 -1 4202560 27 1012 0 0 0 0 0 0 20 0 2 0 337449771 379392000 10485 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 2.88
Current children cumulated vsize (KiB) 683780

[startup+6.30028 s]
/proc/loadavg: 1.55 1.34 1.64 3/183 3837
/proc/meminfo: memFree=26878488/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=370500 CPUtime=1.4 cores=1,3,5,7
/proc/3829/stat : 3829 (python) S 3827 3829 3621 0 -1 4202496 16197 1012 1 0 129 11 0 0 20 0 2 0 337449611 379392000 10485 33554432000 4194304 5683644 140735745319744 140735745314576 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 92625 10485 1894 364 0 27848 0
[pid=3835] ppid=3829 vsize=294720 CPUtime=0 cores=1,3,5,7
/proc/3835/stat : 3835 (python) S 3829 3829 3621 0 -1 4202560 885 0 0 0 0 0 0 0 20 0 1 0 337449771 301793280 8948 33554432000 4194304 5683644 140735745319744 140735745310832 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/3835/statm: 73680 8948 363 364 0 8903 0
[pid=3837] ppid=3835 vsize=30984 CPUtime=4.69 cores=1,3,5,7
/proc/3837/stat : 3837 (clasp-1.3.7-x86) R 3835 3837 3837 0 -1 4202496 8097 0 0 0 467 2 0 0 20 0 1 0 337449771 31727616 7486 33554432000 134512640 136139019 4292475168 4292473776 134831202 0 0 16781312 16386 0 0 0 17 1 0 0 0 0 0
/proc/3837/statm: 7746 7486 181 398 0 7346 0
[pid=3829/tid=3836] ppid=3827 vsize=370500 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3836/stat : 3836 (python) S 3827 3829 3621 0 -1 4202560 27 1012 0 0 0 0 0 0 20 0 2 0 337449771 379392000 10485 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 6.09
Current children cumulated vsize (KiB) 696204

[startup+12.7003 s]
/proc/loadavg: 1.59 1.35 1.64 3/183 3837
/proc/meminfo: memFree=26849908/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=370500 CPUtime=1.4 cores=1,3,5,7
/proc/3829/stat : 3829 (python) S 3827 3829 3621 0 -1 4202496 16197 1012 1 0 129 11 0 0 20 0 2 0 337449611 379392000 10485 33554432000 4194304 5683644 140735745319744 140735745314576 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 92625 10485 1894 364 0 27848 0
[pid=3835] ppid=3829 vsize=294720 CPUtime=0.01 cores=1,3,5,7
/proc/3835/stat : 3835 (python) S 3829 3829 3621 0 -1 4202560 896 0 0 0 0 1 0 0 20 0 1 0 337449771 301793280 8948 33554432000 4194304 5683644 140735745319744 140735745310832 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/3835/statm: 73680 8948 363 364 0 8903 0
[pid=3837] ppid=3835 vsize=40424 CPUtime=11.08 cores=1,3,5,7
/proc/3837/stat : 3837 (clasp-1.3.7-x86) R 3835 3837 3837 0 -1 4202496 10458 0 0 0 1106 2 0 0 20 0 1 0 337449771 41394176 9847 33554432000 134512640 136139019 4292475168 4292473776 134831176 0 0 16781312 16386 0 0 0 17 1 0 0 0 0 0
/proc/3837/statm: 10106 9847 181 398 0 9706 0
[pid=3829/tid=3836] ppid=3827 vsize=370500 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3836/stat : 3836 (python) S 3827 3829 3621 0 -1 4202560 27 1012 0 0 0 0 0 0 20 0 2 0 337449771 379392000 10485 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 12.49
Current children cumulated vsize (KiB) 705644

[startup+25.5003 s]
/proc/loadavg: 1.68 1.38 1.64 3/183 3837
/proc/meminfo: memFree=26790900/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=370500 CPUtime=1.4 cores=1,3,5,7
/proc/3829/stat : 3829 (python) S 3827 3829 3621 0 -1 4202496 16197 1012 1 0 129 11 0 0 20 0 2 0 337449611 379392000 10485 33554432000 4194304 5683644 140735745319744 140735745314576 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 92625 10485 1894 364 0 27848 0
[pid=3835] ppid=3829 vsize=294720 CPUtime=0.03 cores=1,3,5,7
/proc/3835/stat : 3835 (python) S 3829 3829 3621 0 -1 4202560 904 0 0 0 1 2 0 0 20 0 1 0 337449771 301793280 8948 33554432000 4194304 5683644 140735745319744 140735745310832 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 1 0 0 0 0 0
/proc/3835/statm: 73680 8948 363 364 0 8903 0
[pid=3837] ppid=3835 vsize=85604 CPUtime=23.88 cores=1,3,5,7
/proc/3837/stat : 3837 (clasp-1.3.7-x86) R 3835 3837 3837 0 -1 4202496 21736 0 0 0 2384 4 0 0 20 0 1 0 337449771 87658496 21125 33554432000 134512640 136139019 4292475168 4292474000 134832195 0 0 16781312 16386 0 0 0 17 5 0 0 0 0 0
/proc/3837/statm: 21401 21125 181 398 0 21001 0
[pid=3829/tid=3836] ppid=3827 vsize=370500 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3836/stat : 3836 (python) S 3827 3829 3621 0 -1 4202560 27 1012 0 0 0 0 0 0 20 0 2 0 337449771 379392000 10485 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 25.31
Current children cumulated vsize (KiB) 750824

[startup+51.1004 s]
/proc/loadavg: 1.79 1.43 1.65 3/191 3845
/proc/meminfo: memFree=26506180/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=446292 CPUtime=1.43 cores=1,3,5,7
/proc/3829/stat : 3829 (python) S 3827 3829 3621 0 -1 4202496 17149 1012 1 0 131 12 0 0 20 0 3 0 337449611 457003008 10494 33554432000 4194304 5683644 140735745319744 140735745314576 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 111573 10494 1897 364 0 46793 0
[pid=3835] ppid=3829 vsize=370500 CPUtime=0.07 cores=1,3,5,7
/proc/3835/stat : 3835 (python) S 3829 3829 3621 0 -1 4202560 1153 0 0 0 4 3 0 0 20 0 2 0 337449771 379392000 8988 33554432000 4194304 5683644 140735745319744 140735745311392 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3835/statm: 92625 8988 396 364 0 27848 0
[pid=3837] ppid=3835 vsize=114068 CPUtime=47.1 cores=1,3,5,7
/proc/3837/stat : 3837 (clasp-1.3.7-x86) T 3835 3837 3837 0 -1 4202496 28874 0 0 0 4705 5 0 0 20 0 1 0 337449771 116805632 28263 33554432000 134512640 136139019 4292475168 4292473792 134599813 0 0 16781312 16386 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/3837/statm: 28517 28263 181 398 0 28117 0
[pid=3835/tid=3842] ppid=3829 vsize=370500 CPUtime=0 cores=1,3,5,7
/proc/3835/task/3842/stat : 3842 (python) S 3829 3829 3621 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 337454484 379392000 8988 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=3843] ppid=3829 vsize=370512 CPUtime=0 cores=1,3,5,7
/proc/3843/stat : 3843 (python) S 3829 3829 3621 0 -1 4202560 789 0 0 0 0 0 0 0 20 0 1 0 337454487 379404288 8926 33554432000 4194304 5683644 140735745319744 140735745310832 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/3843/statm: 92628 8926 335 364 0 27848 0
[pid=3845] ppid=3843 vsize=156784 CPUtime=2.32 cores=1,3,5,7
/proc/3845/stat : 3845 (scip-2.0.1.linu) R 3843 3845 3845 0 -1 4202496 36828 0 0 0 226 6 0 0 20 0 1 0 337454487 160546816 35789 33554432000 4194304 13684479 140734016547968 140734016541896 11876362 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/3845/statm: 39196 35789 742 2317 0 36874 0
[pid=3829/tid=3836] ppid=3827 vsize=446292 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3836/stat : 3836 (python) S 3827 3829 3621 0 -1 4202560 27 1012 0 0 0 0 0 0 20 0 3 0 337449771 457003008 10494 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=3829/tid=3844] ppid=3827 vsize=446292 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3844/stat : 3844 (python) S 3827 3829 3621 0 -1 4202560 21 1012 0 0 0 0 0 0 20 0 3 0 337454487 457003008 10494 33554432000 4194304 5683644 140735745319744 140730609488432 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 50.92
Current children cumulated vsize (KiB) 1458156

[startup+102.301 s]
/proc/loadavg: 1.91 1.51 1.66 3/199 3853
/proc/meminfo: memFree=26214228/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=522084 CPUtime=1.46 cores=1,3,5,7
/proc/3829/stat : 3829 (python) S 3827 3829 3621 0 -1 4202496 18018 1012 1 0 134 12 0 0 20 0 4 0 337449611 534614016 10510 33554432000 4194304 5683644 140735745319744 140735745314576 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 130521 10510 1907 364 0 65738 0
[pid=3835] ppid=3829 vsize=370500 CPUtime=0.07 cores=1,3,5,7
/proc/3835/stat : 3835 (python) S 3829 3829 3621 0 -1 4202560 1153 0 0 0 4 3 0 0 20 0 2 0 337449771 379392000 8988 33554432000 4194304 5683644 140735745319744 140735745311392 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3835/statm: 92625 8988 396 364 0 27848 0
[pid=3837] ppid=3835 vsize=114068 CPUtime=47.1 cores=1,3,5,7
/proc/3837/stat : 3837 (clasp-1.3.7-x86) T 3835 3837 3837 0 -1 4202496 28874 0 0 0 4705 5 0 0 20 0 1 0 337449771 116805632 28263 33554432000 134512640 136139019 4292475168 4292473792 134599813 0 0 16781312 16386 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/3837/statm: 28517 28263 181 398 0 28117 0
[pid=3835/tid=3842] ppid=3829 vsize=370500 CPUtime=0 cores=1,3,5,7
/proc/3835/task/3842/stat : 3842 (python) S 3829 3829 3621 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 337454484 379392000 8988 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=3843] ppid=3829 vsize=370512 CPUtime=0.07 cores=1,3,5,7
/proc/3843/stat : 3843 (python) S 3829 3829 3621 0 -1 4202560 1090 0 0 0 4 3 0 0 20 0 2 0 337454487 379404288 8975 33554432000 4194304 5683644 140735745319744 140735745311392 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/3843/statm: 92628 8975 383 364 0 27848 0
[pid=3845] ppid=3843 vsize=153280 CPUtime=47.74 cores=1,3,5,7
/proc/3845/stat : 3845 (scip-2.0.1.linu) T 3843 3845 3845 0 -1 4202496 39024 0 0 0 4758 16 0 0 20 0 1 0 337454487 156958720 35766 33554432000 4194304 13684479 140734016547968 140734016539880 7452957 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/3845/statm: 38320 35766 757 2317 0 35998 0
[pid=3843/tid=3850] ppid=3829 vsize=370512 CPUtime=0 cores=1,3,5,7
/proc/3843/task/3850/stat : 3850 (python) S 3829 3829 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337459263 379404288 8975 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=3851] ppid=3829 vsize=446304 CPUtime=0 cores=1,3,5,7
/proc/3851/stat : 3851 (python) S 3829 3829 3621 0 -1 4202560 920 0 0 0 0 0 0 0 20 0 1 0 337459266 457015296 8955 33554432000 4194304 5683644 140735745319744 140735745310832 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 1 0 0 0 0 0
/proc/3851/statm: 111576 8955 358 364 0 46793 0
[pid=3853] ppid=3851 vsize=158140 CPUtime=5.73 cores=1,3,5,7
/proc/3853/stat : 3853 (scip-2.0.1.linu) R 3851 3853 3853 0 -1 4202496 37131 0 0 0 566 7 0 0 20 0 1 0 337459266 161935360 36438 33554432000 4194304 15243407 140733295175712 140733295171400 13277498 0 0 16781312 2 0 0 0 17 3 0 0 0 0 0
/proc/3853/statm: 39535 36438 902 2698 0 36830 0
[pid=3829/tid=3836] ppid=3827 vsize=522084 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3836/stat : 3836 (python) S 3827 3829 3621 0 -1 4202560 27 1012 0 0 0 0 0 0 20 0 4 0 337449771 534614016 10510 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=3829/tid=3844] ppid=3827 vsize=522084 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3844/stat : 3844 (python) S 3827 3829 3621 0 -1 4202560 21 1012 0 0 0 0 0 0 20 0 4 0 337454487 534614016 10510 33554432000 4194304 5683644 140735745319744 140730609488432 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0

################
# More data... #
################

[pid=3919/tid=3935] ppid=3917 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/3919/task/3935/stat : 3935 (java) T 3917 3919 3919 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 337606818 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492285836280 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=3919/tid=3936] ppid=3917 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/3919/task/3936/stat : 3936 (java) T 3917 3919 3919 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 337606818 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492284783944 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3919/tid=3937] ppid=3917 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/3919/task/3937/stat : 3937 (java) T 3917 3919 3919 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 337606878 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492283728680 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3917/tid=3938] ppid=3829 vsize=749472 CPUtime=0 cores=1,3,5,7
/proc/3917/task/3938/stat : 3938 (python) S 3829 3829 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337611337 767459328 9513 33554432000 4194304 5683644 140735745319744 140730364318256 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=3829/tid=3836] ppid=3827 vsize=825252 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3836/stat : 3836 (python) S 3827 3829 3621 0 -1 4202560 36 1012 0 0 0 0 0 0 20 0 8 0 337449771 845058048 11050 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=3829/tid=3844] ppid=3827 vsize=825252 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3844/stat : 3844 (python) S 3827 3829 3621 0 -1 4202560 30 1012 0 0 0 0 0 0 20 0 8 0 337454487 845058048 11050 33554432000 4194304 5683644 140735745319744 140730609488432 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=3829/tid=3852] ppid=3827 vsize=825252 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3852/stat : 3852 (python) S 3827 3829 3621 0 -1 4202560 25 1012 0 0 0 0 0 0 20 0 8 0 337459266 845058048 11050 33554432000 4194304 5683644 140735745319744 140730598998576 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=3829/tid=3864] ppid=3827 vsize=825252 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3864/stat : 3864 (python) S 3827 3829 3621 0 -1 4202560 21 1012 0 0 0 0 0 0 20 0 8 0 337473403 845058048 11050 33554432000 4194304 5683644 140735745319744 140730588508720 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=3829/tid=3901] ppid=3827 vsize=825252 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3901/stat : 3901 (python) S 3827 3829 3621 0 -1 4202560 19 1012 0 0 0 0 0 0 20 0 8 0 337537438 845058048 11050 33554432000 4194304 5683644 140735745319744 140730374808112 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=3829/tid=3914] ppid=3827 vsize=825252 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3914/stat : 3914 (python) S 3827 3829 3621 0 -1 4202560 22 1012 0 0 0 0 0 0 20 0 8 0 337601952 845058048 11050 33554432000 4194304 5683644 140735745319744 140730364318256 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=3829/tid=3918] ppid=3827 vsize=825252 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3918/stat : 3918 (python) S 3827 3829 3621 0 -1 4202560 22 1012 0 0 0 0 0 0 20 0 8 0 337606811 845058048 11050 33554432000 4194304 5683644 140735745319744 140730353828400 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1665.91
Current children cumulated vsize (KiB) 15577464



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1708.5 s]
/proc/loadavg: 2.00 2.01 1.91 3/250 3943
/proc/meminfo: memFree=23984332/32873844 swapFree=6608/6608
[pid=3829] ppid=3827 vsize=901044 CPUtime=4.36 cores=1,3,5,7
/proc/3829/stat : 3829 (python) S 3827 3829 3621 0 -1 4202496 23050 1012 1 0 420 16 0 0 20 0 9 0 337449611 922669056 11058 33554432000 4194304 5683644 140735745319744 140735745314576 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3829/statm: 225261 11058 1917 364 0 160463 0
[pid=3835] ppid=3829 vsize=370500 CPUtime=0.5 cores=1,3,5,7
/proc/3835/stat : 3835 (python) S 3829 3829 3621 0 -1 4202560 1230 0 0 0 30 20 0 0 20 0 2 0 337449771 379392000 8989 33554432000 4194304 5683644 140735745319744 140735745311392 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/3835/statm: 92625 8989 397 364 0 27848 0
[pid=3837] ppid=3835 vsize=162880 CPUtime=274.72 cores=1,3,5,7
/proc/3837/stat : 3837 (clasp-1.3.7-x86) T 3835 3837 3837 0 -1 4202496 41411 0 0 0 27463 9 0 0 20 0 1 0 337449771 166789120 40474 33554432000 134512640 136139019 4292475168 4292474032 134608362 0 0 16781312 16386 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/3837/statm: 40720 40474 181 398 0 40320 0
[pid=3835/tid=3842] ppid=3829 vsize=370500 CPUtime=0 cores=1,3,5,7
/proc/3835/task/3842/stat : 3842 (python) S 3829 3829 3621 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 337454484 379392000 8989 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=3843] ppid=3829 vsize=370512 CPUtime=1.89 cores=1,3,5,7
/proc/3843/stat : 3843 (python) S 3829 3829 3621 0 -1 4202560 1196 0 0 0 103 86 0 0 20 0 2 0 337454487 379404288 8975 33554432000 4194304 5683644 140735745319744 140735745311392 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3843/statm: 92628 8975 383 364 0 27848 0
[pid=3845] ppid=3843 vsize=221176 CPUtime=1052.09 cores=1,3,5,7
/proc/3845/stat : 3845 (scip-2.0.1.linu) T 3843 3845 3845 0 -1 4202496 56611 0 0 0 105067 142 0 0 20 0 1 0 337454487 226484224 52649 33554432000 4194304 13684479 140734016547968 140734016541896 7251528 0 0 16781312 2 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/3845/statm: 55294 52649 825 2317 0 52972 0
[pid=3843/tid=3850] ppid=3829 vsize=370512 CPUtime=0 cores=1,3,5,7
/proc/3843/task/3850/stat : 3850 (python) S 3829 3829 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337459263 379404288 8975 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=3851] ppid=3829 vsize=446304 CPUtime=0.25 cores=1,3,5,7
/proc/3851/stat : 3851 (python) S 3829 3829 3621 0 -1 4202560 1154 0 0 0 14 11 0 0 20 0 2 0 337459266 457015296 8981 33554432000 4194304 5683644 140735745319744 140735745311392 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/3851/statm: 111576 8981 383 364 0 46793 0
[pid=3853] ppid=3851 vsize=161984 CPUtime=138.58 cores=1,3,5,7
/proc/3853/stat : 3853 (scip-2.0.1.linu) T 3851 3853 3853 0 -1 4202496 150427 0 0 0 13817 41 0 0 20 0 1 0 337459266 165871616 38085 33554432000 4194304 15243407 140733295175712 140733295170104 7127982 0 0 16781312 2 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/3853/statm: 40496 38085 955 2698 0 37791 0
[pid=3851/tid=3855] ppid=3829 vsize=446304 CPUtime=0 cores=1,3,5,7
/proc/3851/task/3855/stat : 3855 (python) S 3829 3829 3621 0 -1 4202560 51 0 0 0 0 0 0 0 20 0 2 0 337464202 457015296 8981 33554432000 4194304 5683644 140735745319744 140730609488432 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=3863] ppid=3829 vsize=522096 CPUtime=0.07 cores=1,3,5,7
/proc/3863/stat : 3863 (python) S 3829 3829 3621 0 -1 4202560 1141 0 0 0 4 3 0 0 20 0 2 0 337473403 534626304 8987 33554432000 4194304 5683644 140735745319744 140735745311392 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/3863/statm: 130524 8987 383 364 0 65738 0
[pid=3865] ppid=3863 vsize=123912 CPUtime=47.11 cores=1,3,5,7
/proc/3865/stat : 3865 (minion) T 3863 3865 3865 0 -1 4202496 19559 0 0 0 4708 3 0 0 20 0 1 0 337473403 126885888 23677 33554432000 134512640 165565024 4294706640 4294703888 142765053 0 0 16781312 8396802 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/3865/statm: 30978 23677 332 7582 0 23393 0
[pid=3863/tid=3867] ppid=3829 vsize=522096 CPUtime=0 cores=1,3,5,7
/proc/3863/task/3867/stat : 3867 (python) S 3829 3829 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337478117 534626304 8987 33554432000 4194304 5683644 140735745319744 140730598998576 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=3900] ppid=3829 vsize=597888 CPUtime=0.18 cores=1,3,5,7
/proc/3900/stat : 3900 (python) S 3829 3829 3621 0 -1 4202560 1185 0 0 0 9 9 0 0 20 0 2 0 337537438 612237312 8993 33554432000 4194304 5683644 140735745319744 140735745311392 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3900/statm: 149472 8993 383 364 0 84683 0
[pid=3902] ppid=3900 vsize=602068 CPUtime=96.31 cores=1,3,5,7
/proc/3902/stat : 3902 (bsolo_pb10) T 3900 3902 3902 0 -1 4202496 151394 0 0 0 9487 144 0 0 20 0 1 0 337537438 616517632 148040 33554432000 134512640 137138936 4292948384 4292947312 134542187 0 0 16781312 16384 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/3902/statm: 150517 148040 190 642 0 149873 0
[pid=3900/tid=3906] ppid=3829 vsize=597888 CPUtime=0 cores=1,3,5,7
/proc/3900/task/3906/stat : 3906 (python) S 3829 3829 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337547072 612237312 8993 33554432000 4194304 5683644 140735745319744 140730588508720 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=3913] ppid=3829 vsize=673680 CPUtime=0.08 cores=1,3,5,7
/proc/3913/stat : 3913 (python) S 3829 3829 3621 0 -1 4202560 1151 0 0 0 5 3 0 0 20 0 2 0 337601952 689848320 9506 33554432000 4194304 5683644 140735745319744 140735745311392 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/3913/statm: 168420 9506 383 364 0 103628 0
[pid=3915] ppid=3913 vsize=124768 CPUtime=48.55 cores=1,3,5,7
/proc/3915/stat : 3915 (wbo1.4a) T 3913 3915 3915 0 -1 4202496 31424 0 0 0 4844 11 0 0 20 0 1 0 337601953 127762432 30618 33554432000 134512640 135751879 4289559856 4289557472 134605088 0 0 16781312 3 18446744071579387992 0 0 17 3 0 0 0 0 0
/proc/3915/statm: 31192 30618 143 303 0 30887 0
[pid=3913/tid=3916] ppid=3829 vsize=673680 CPUtime=0 cores=1,3,5,7
/proc/3913/task/3916/stat : 3916 (python) S 3829 3829 3621 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 337606810 689848320 9506 33554432000 4194304 5683644 140735745319744 140730374808112 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=3917] ppid=3829 vsize=749472 CPUtime=0.08 cores=1,3,5,7
/proc/3917/stat : 3917 (python) S 3829 3829 3621 0 -1 4202560 1172 0 0 0 4 4 0 0 20 0 2 0 337606811 767459328 9513 33554432000 4194304 5683644 140735745319744 140735745311392 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/3917/statm: 187368 9513 383 364 0 122573 0
[pid=3919] ppid=3917 vsize=9624972 CPUtime=46.51 cores=1,3,5,7
/proc/3919/stat : 3919 (java) T 3917 3919 3919 0 -1 4202496 15490 0 1 0 4638 13 0 0 20 0 19 0 337606811 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140735327457376 213422997565 0 0 0 16800975 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/3919/statm: 2406243 62518 2267 9 0 2396937 0
[pid=3919/tid=3920] ppid=3917 vsize=9624972 CPUtime=44.87 cores=1,3,5,7
/proc/3919/task/3920/stat : 3920 (java) T 3917 3919 3919 0 -1 4202560 2557 0 1 0 4484 3 0 0 20 0 19 0 337606813 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492468019488 140492408461610 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3919/tid=3921] ppid=3917 vsize=9624972 CPUtime=0.01 cores=1,3,5,7
/proc/3919/task/3921/stat : 3921 (java) T 3917 3919 3919 0 -1 4202560 15 0 0 0 1 0 0 0 20 0 19 0 337606814 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492311931048 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3919/tid=3922] ppid=3917 vsize=9624972 CPUtime=0.03 cores=1,3,5,7
/proc/3919/task/3922/stat : 3922 (java) T 3917 3919 3919 0 -1 4202560 27 0 0 0 3 0 0 0 20 0 19 0 337606814 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492310878504 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3919/tid=3923] ppid=3917 vsize=9624972 CPUtime=0.01 cores=1,3,5,7
/proc/3919/task/3923/stat : 3923 (java) T 3917 3919 3919 0 -1 4202560 68 0 0 0 1 0 0 0 20 0 19 0 337606814 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492309825960 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=3919/tid=3924] ppid=3917 vsize=9624972 CPUtime=0.02 cores=1,3,5,7
/proc/3919/task/3924/stat : 3924 (java) T 3917 3919 3919 0 -1 4202560 110 0 0 0 2 0 0 0 20 0 19 0 337606814 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492308773416 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3919/tid=3925] ppid=3917 vsize=9624972 CPUtime=0.03 cores=1,3,5,7
/proc/3919/task/3925/stat : 3925 (java) T 3917 3919 3919 0 -1 4202560 124 0 0 0 2 1 0 0 20 0 19 0 337606814 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492307720872 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3919/tid=3926] ppid=3917 vsize=9624972 CPUtime=0.02 cores=1,3,5,7
/proc/3919/task/3926/stat : 3926 (java) T 3917 3919 3919 0 -1 4202560 17 0 0 0 2 0 0 0 20 0 19 0 337606814 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492306668328 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3919/tid=3927] ppid=3917 vsize=9624972 CPUtime=0.01 cores=1,3,5,7
/proc/3919/task/3927/stat : 3927 (java) T 3917 3919 3919 0 -1 4202560 6 0 0 0 1 0 0 0 20 0 19 0 337606814 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492305615784 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=3919/tid=3928] ppid=3917 vsize=9624972 CPUtime=0.04 cores=1,3,5,7
/proc/3919/task/3928/stat : 3928 (java) T 3917 3919 3919 0 -1 4202560 125 0 0 0 3 1 0 0 20 0 19 0 337606814 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492304563240 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=3919/tid=3929] ppid=3917 vsize=9624972 CPUtime=0.12 cores=1,3,5,7
/proc/3919/task/3929/stat : 3929 (java) T 3917 3919 3919 0 -1 4202560 117 0 0 0 12 0 0 0 20 0 19 0 337606815 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492292151400 213423011753 0 0 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3919/tid=3930] ppid=3917 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/3919/task/3930/stat : 3930 (java) T 3917 3919 3919 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 337606816 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492291097720 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3919/tid=3931] ppid=3917 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/3919/task/3931/stat : 3931 (java) T 3917 3919 3919 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 337606816 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492290045064 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=3919/tid=3932] ppid=3917 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/3919/task/3932/stat : 3932 (java) T 3917 3919 3919 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 337606818 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492288993808 213423019808 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3919/tid=3933] ppid=3917 vsize=9624972 CPUtime=0.62 cores=1,3,5,7
/proc/3919/task/3933/stat : 3933 (java) T 3917 3919 3919 0 -1 4202560 7153 0 0 0 60 2 0 0 20 0 19 0 337606818 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492287941176 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=3919/tid=3934] ppid=3917 vsize=9624972 CPUtime=0.45 cores=1,3,5,7
/proc/3919/task/3934/stat : 3934 (java) T 3917 3919 3919 0 -1 4202560 3745 0 0 0 43 2 0 0 20 0 19 0 337606818 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492286888632 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=3919/tid=3935] ppid=3917 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/3919/task/3935/stat : 3935 (java) T 3917 3919 3919 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 337606818 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492285836280 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=3919/tid=3936] ppid=3917 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/3919/task/3936/stat : 3936 (java) T 3917 3919 3919 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 337606818 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492284783944 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3919/tid=3937] ppid=3917 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/3919/task/3937/stat : 3937 (java) T 3917 3919 3919 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 337606878 9855971328 62518 33554432000 1073741824 1073778376 140735327466240 140492283728680 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=3917/tid=3938] ppid=3829 vsize=749472 CPUtime=0 cores=1,3,5,7
/proc/3917/task/3938/stat : 3938 (python) S 3829 3829 3621 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 337611337 767459328 9513 33554432000 4194304 5683644 140735745319744 140730364318256 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=3941] ppid=3829 vsize=825264 CPUtime=0 cores=1,3,5,7
/proc/3941/stat : 3941 (python) S 3829 3829 3621 0 -1 4202560 867 0 0 0 0 0 0 0 20 0 1 0 337620368 845070336 9470 33554432000 4194304 5683644 140735745319744 140735745310832 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/3941/statm: 206316 9470 335 364 0 141518 0
[pid=3943] ppid=3941 vsize=150660 CPUtime=0.92 cores=1,3,5,7
/proc/3943/stat : 3943 (pbct-0.1.2-linu) R 3941 3943 3943 0 -1 4202496 39472 0 0 0 86 6 0 0 20 0 1 0 337620369 154275840 36244 33554432000 134512640 136422188 4292714240 4292713176 135811945 0 0 16781312 16387 0 0 0 17 1 0 0 0 0 0
/proc/3943/statm: 37665 36245 192 467 0 37196 0
[pid=3829/tid=3836] ppid=3827 vsize=901044 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3836/stat : 3836 (python) S 3827 3829 3621 0 -1 4202560 41 1012 0 0 0 0 0 0 20 0 9 0 337449771 922669056 11058 33554432000 4194304 5683644 140735745319744 140730624320048 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=3829/tid=3844] ppid=3827 vsize=901044 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3844/stat : 3844 (python) S 3827 3829 3621 0 -1 4202560 30 1012 0 0 0 0 0 0 20 0 9 0 337454487 922669056 11058 33554432000 4194304 5683644 140735745319744 140730609488432 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=3829/tid=3852] ppid=3827 vsize=901044 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3852/stat : 3852 (python) S 3827 3829 3621 0 -1 4202560 25 1012 0 0 0 0 0 0 20 0 9 0 337459266 922669056 11058 33554432000 4194304 5683644 140735745319744 140730598998576 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=3829/tid=3864] ppid=3827 vsize=901044 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3864/stat : 3864 (python) S 3827 3829 3621 0 -1 4202560 21 1012 0 0 0 0 0 0 20 0 9 0 337473403 922669056 11058 33554432000 4194304 5683644 140735745319744 140730588508720 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=3829/tid=3901] ppid=3827 vsize=901044 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3901/stat : 3901 (python) S 3827 3829 3621 0 -1 4202560 19 1012 0 0 0 0 0 0 20 0 9 0 337537438 922669056 11058 33554432000 4194304 5683644 140735745319744 140730374808112 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=3829/tid=3914] ppid=3827 vsize=901044 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3914/stat : 3914 (python) S 3827 3829 3621 0 -1 4202560 22 1012 0 0 0 0 0 0 20 0 9 0 337601952 922669056 11058 33554432000 4194304 5683644 140735745319744 140730364318256 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=3829/tid=3918] ppid=3827 vsize=901044 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3918/stat : 3918 (python) S 3827 3829 3621 0 -1 4202560 22 1012 0 0 0 0 0 0 20 0 9 0 337606811 922669056 11058 33554432000 4194304 5683644 140735745319744 140730353828400 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=3829/tid=3942] ppid=3827 vsize=901044 CPUtime=0 cores=1,3,5,7
/proc/3829/task/3942/stat : 3942 (python) S 3827 3829 3621 0 -1 4202560 22 1012 0 0 0 0 0 0 20 0 9 0 337620368 922669056 11058 33554432000 4194304 5683644 140735745319744 140730343338544 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1712.2
Current children cumulated vsize (KiB) 16629180

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 3829 and gives
#  childrusage.ru_utime.tv_sec=4
#  childrusage.ru_utime.tv_usec=205360
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=163975
# CPU time returned by wait4() is 4.36934
# while last known CPU time is 1712.2
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1708.57
CPU time (s): 1712.2
CPU user time (s): 1706.99
CPU system time (s): 5.21
CPU usage (%): 100.213
Max. virtual memory (cumulated for all children) (KiB): 16629180

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4.20536
system time used= 0.163975
maximum resident set size= 44944
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 24062
page faults= 1
swaps= 0
block input operations= 0
block output operations= 5608
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1154
involuntary context switches= 45

runsolver used 7.47086 second user time and 16.4125 second system time

The end

Launcher Data 

Begin job on node140 at 2012-06-03 10:47:27
IDJOB=3728947
IDBENCH=1155
IDSOLVER=2327
FILE ID=node140/3728947-1338713247
RUNJOBID= node140-1338713222-3638
PBS_JOBID= 14636725
Free space on /tmp= 70524 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB06//final/normalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/prestwich/robin/normalized-robin18.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3728947-1338713247/watcher-3728947-1338713247 -o /tmp/evaluation-result-3728947-1338713247/solver-3728947-1338713247 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3728947-1338713247.opb -seed 808678085 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f17e31cb6762bd0e8d3e659d7bd1050c
RANDOM SEED=808678085

node140.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        26951556 kB
Buffers:          417428 kB
Cached:          4547864 kB
SwapCached:         1664 kB
Active:          1623792 kB
Inactive:        3397588 kB
Active(anon):      48776 kB
Inactive(anon):     9408 kB
Active(file):    1575016 kB
Inactive(file):  3388180 kB
Unevictable:        6608 kB
Mlocked:            6608 kB
SwapTotal:      67108856 kB
SwapFree:       67105156 kB
Dirty:            208360 kB
Writeback:            16 kB
AnonPages:         61292 kB
Mapped:            17196 kB
Shmem:                36 kB
Slab:             760212 kB
SReclaimable:     200304 kB
SUnreclaim:       559908 kB
KernelStack:        1488 kB
PageTables:         5204 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     215588 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70416 MiB
End job on node140 at 2012-06-03 11:15:58