Trace number 3742655

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 1078.27 1075.97

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/ShortestPathBA/
normalized-BeauxArts_K66.opb
MD5SUM03e282ee85097a100b1dfa3533d9978b
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark871.544
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables3630
Total number of constraints3764
Number of constraints which are clauses3698
Number of constraints which are cardinality constraints (but not clauses)66
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint66
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 67
Number of bits of the biggest sum of numbers7
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.40/1.84	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.40/1.84	c INFO: solving HOME/instance-3742655-1338725341.opb
0.47/1.88	c INFO: parsing took 0.04 s
0.51/1.96	c INFO: features took 0.07 CPU seconds
0.51/1.99	c INFO: running scip-2.0.1-clp@0 for 45 with 1799 remaining (b = 0.72)
50.45/51.84	c INFO: running clasp-1.3.7@0 for 45 with 1750 remaining (b = 0.37)
97.57/98.99	c INFO: running scip-2.0.1-spx@0 for 45 with 1703 remaining (b = 0.07)
145.94/147.25	c INFO: running clasp-1.3.7@47 for 45 with 1655 remaining (b = 0.01)
191.54/192.72	c INFO: wrote minion input file to HOME/tmp4yrpC_.minion
191.54/192.72	c INFO: running minion-0.12@0 for 45 with 1610 remaining (b = 0.10)
238.88/240.01	c INFO: running pbct-0.1.2-linear@0 for 45 with 1562 remaining (b = 0.02)
290.11/291.22	c INFO: running scip-2.0.1-clp@49 for 45 with 1515 remaining (b = 0.01)
335.29/336.37	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1470 remaining (b = 0.05)
385.10/383.87	c INFO: running clasp-1.3.7@92 for 136 with 1421 remaining (b = 0.02)
522.71/521.27	c INFO: running scip-2.0.1-spx@47 for 45 with 1284 remaining (b = 0.01)
567.97/566.43	c INFO: running bsolo_pb10-l2@0 for 45 with 1239 remaining (b = 0.02)
615.63/614.09	c INFO: running scip-2.0.1-spx@92 for 410 with 1191 remaining (b = 0.06)
1029.92/1027.64	c INFO: running bsolo_pb10-l3@0 for 45 with 779 remaining (b = 0.01)
1077.56/1075.29	c INFO: running bsolo_pb10-l1@0 for 91 with 732 remaining (b = 0.01)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742655-1338725341/watcher-3742655-1338725341 -o /tmp/evaluation-result-3742655-1338725341/solver-3742655-1338725341 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742655-1338725341.opb -seed 1222798511 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.99 1.99 2.07 3/187 7452
/proc/meminfo: memFree=26333856/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=15276 CPUtime=0 cores=0,2,4,6
/proc/7452/stat : 7452 (python) D 7450 7452 3621 0 -1 4202496 634 0 0 0 0 0 0 0 20 0 1 0 338658624 15642624 527 33554432000 4194304 5683644 140735751876304 140735751862216 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/7452/statm: 3819 527 312 364 0 261 0

[startup+0.0993709 s]
/proc/loadavg: 1.99 1.99 2.07 3/187 7452
/proc/meminfo: memFree=26333856/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=19084 CPUtime=0.02 cores=0,2,4,6
/proc/7452/stat : 7452 (python) R 7450 7452 3621 0 -1 4202496 1392 0 0 0 2 0 0 0 20 0 1 0 338658624 19542016 1146 33554432000 4194304 5683644 140735751876304 140735751806808 4875555 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7452/statm: 4771 1146 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.100219 s]
/proc/loadavg: 1.99 1.99 2.07 3/187 7452
/proc/meminfo: memFree=26333856/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=19084 CPUtime=0.02 cores=0,2,4,6
/proc/7452/stat : 7452 (python) R 7450 7452 3621 0 -1 4202496 1393 0 0 0 2 0 0 0 20 0 1 0 338658624 19542016 1147 33554432000 4194304 5683644 140735751876304 140735751806808 4591888 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7452/statm: 4771 1147 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.300228 s]
/proc/loadavg: 1.99 1.99 2.07 3/187 7452
/proc/meminfo: memFree=26333856/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=96516 CPUtime=0.11 cores=0,2,4,6
/proc/7452/stat : 7452 (python) D 7450 7452 3621 0 -1 4202496 3736 1011 1 0 9 2 0 0 20 0 1 0 338658624 98832384 2550 33554432000 4194304 5683644 140735751876304 140735751812168 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/7452/statm: 24129 2550 822 364 0 1620 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 96516

[startup+0.70018 s]
/proc/loadavg: 1.99 1.99 2.07 3/187 7452
/proc/meminfo: memFree=26333856/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=122348 CPUtime=0.23 cores=0,2,4,6
/proc/7452/stat : 7452 (python) D 7450 7452 3621 0 -1 4202496 5975 1011 1 0 16 7 0 0 20 0 1 0 338658624 125284352 3850 33554432000 4194304 5683644 140735751876304 140735751787464 213419639696 0 0 16781312 2 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/7452/statm: 30587 3850 1015 364 0 2830 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 122348

[startup+1.50025 s]
/proc/loadavg: 1.99 1.99 2.07 2/188 7457
/proc/meminfo: memFree=26313512/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=260020 CPUtime=0.4 cores=0,2,4,6
/proc/7452/stat : 7452 (python) D 7450 7452 3621 0 -1 4202496 9117 1011 1 0 29 11 0 0 20 0 1 0 338658624 266260480 6410 33554432000 4194304 5683644 140735751876304 140735751826024 213419644775 0 0 16781312 2 18446744071580543392 0 0 17 0 0 0 0 0 0
/proc/7452/statm: 65005 6410 1706 364 0 5057 0
Current children cumulated CPU time (s) 0.4
Current children cumulated vsize (KiB) 260020

[startup+3.10035 s]
/proc/loadavg: 1.99 1.99 2.07 3/191 7460
/proc/meminfo: memFree=26257068/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=358872 CPUtime=0.59 cores=0,2,4,6
/proc/7452/stat : 7452 (python) S 7450 7452 3621 0 -1 4202496 10894 1011 1 0 47 12 0 0 20 0 2 0 338658624 367484928 7575 33554432000 4194304 5683644 140735751876304 140735751871136 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 54 0 0
/proc/7452/statm: 89718 7575 1900 364 0 24941 0
[pid=7458] ppid=7452 vsize=283092 CPUtime=0 cores=0,2,4,6
/proc/7458/stat : 7458 (python) S 7452 7452 3621 0 -1 4202560 767 0 0 0 0 0 0 0 20 0 1 0 338658823 289886208 6010 33554432000 4194304 5683644 140735751876304 140735751867392 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 4 0 0 0 0 0
/proc/7458/statm: 70773 6010 341 364 0 5996 0
[pid=7460] ppid=7458 vsize=71732 CPUtime=1.09 cores=0,2,4,6
/proc/7460/stat : 7460 (scip-2.0.1.linu) R 7458 7460 7460 0 -1 4202496 15647 0 0 0 104 5 0 0 20 0 1 0 338658823 73453568 15045 33554432000 4194304 15243407 140735760118528 140735760114840 6144933 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7460/statm: 17933 15045 689 2698 0 15228 0
[pid=7452/tid=7459] ppid=7450 vsize=358872 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7459/stat : 7459 (python) S 7450 7452 3621 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 2 0 338658823 367484928 7575 33554432000 4194304 5683644 140735751876304 139881587155504 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1.68
Current children cumulated vsize (KiB) 713696
heavy processes:

[startup+6.30036 s]
/proc/loadavg: 1.99 1.99 2.07 3/191 7460
/proc/meminfo: memFree=26203252/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=358872 CPUtime=0.59 cores=0,2,4,6
/proc/7452/stat : 7452 (python) S 7450 7452 3621 0 -1 4202496 10894 1011 1 0 47 12 0 0 20 0 2 0 338658624 367484928 7575 33554432000 4194304 5683644 140735751876304 140735751871136 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 54 0 0
/proc/7452/statm: 89718 7575 1900 364 0 24941 0
[pid=7458] ppid=7452 vsize=283092 CPUtime=0 cores=0,2,4,6
/proc/7458/stat : 7458 (python) S 7452 7452 3621 0 -1 4202560 893 0 0 0 0 0 0 0 20 0 1 0 338658823 289886208 6033 33554432000 4194304 5683644 140735751876304 140735751867392 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/7458/statm: 70773 6033 364 364 0 5996 0
[pid=7460] ppid=7458 vsize=106880 CPUtime=4.3 cores=0,2,4,6
/proc/7460/stat : 7460 (scip-2.0.1.linu) R 7458 7460 7460 0 -1 4202496 24310 0 0 0 420 10 0 0 20 0 1 0 338658823 109445120 23708 33554432000 4194304 15243407 140735760118528 140735760114840 4559432 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7460/statm: 26720 23708 693 2698 0 24015 0
[pid=7452/tid=7459] ppid=7450 vsize=358872 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7459/stat : 7459 (python) S 7450 7452 3621 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 2 0 338658823 367484928 7575 33554432000 4194304 5683644 140735751876304 139881587155504 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 4.89
Current children cumulated vsize (KiB) 748844
heavy processes:

[startup+12.7003 s]
/proc/loadavg: 1.99 1.99 2.07 3/192 7461
/proc/meminfo: memFree=26146632/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=358872 CPUtime=0.59 cores=0,2,4,6
/proc/7452/stat : 7452 (python) S 7450 7452 3621 0 -1 4202496 10894 1011 1 0 47 12 0 0 20 0 2 0 338658624 367484928 7575 33554432000 4194304 5683644 140735751876304 140735751871136 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 54 0 0
/proc/7452/statm: 89718 7575 1900 364 0 24941 0
[pid=7458] ppid=7452 vsize=283092 CPUtime=0.01 cores=0,2,4,6
/proc/7458/stat : 7458 (python) S 7452 7452 3621 0 -1 4202560 984 0 0 0 0 1 0 0 20 0 1 0 338658823 289886208 6038 33554432000 4194304 5683644 140735751876304 140735751867392 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/7458/statm: 70773 6038 369 364 0 5996 0
[pid=7460] ppid=7458 vsize=138924 CPUtime=10.69 cores=0,2,4,6
/proc/7460/stat : 7460 (scip-2.0.1.linu) R 7458 7460 7460 0 -1 4202496 32024 0 0 0 1037 32 0 0 20 0 1 0 338658823 142258176 31396 33554432000 4194304 15243407 140735760118528 140735760114600 13354455 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7460/statm: 34731 31396 888 2698 0 32026 0
[pid=7452/tid=7459] ppid=7450 vsize=358872 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7459/stat : 7459 (python) S 7450 7452 3621 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 2 0 338658823 367484928 7575 33554432000 4194304 5683644 140735751876304 139881587155504 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 11.29
Current children cumulated vsize (KiB) 780888

[startup+25.5065 s]
/proc/loadavg: 1.99 1.99 2.07 3/191 7461
/proc/meminfo: memFree=26081632/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=358872 CPUtime=0.59 cores=0,2,4,6
/proc/7452/stat : 7452 (python) S 7450 7452 3621 0 -1 4202496 10894 1011 1 0 47 12 0 0 20 0 2 0 338658624 367484928 7575 33554432000 4194304 5683644 140735751876304 140735751871136 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 54 0 0
/proc/7452/statm: 89718 7575 1900 364 0 24941 0
[pid=7458] ppid=7452 vsize=283092 CPUtime=0.03 cores=0,2,4,6
/proc/7458/stat : 7458 (python) S 7452 7452 3621 0 -1 4202560 997 0 0 0 1 2 0 0 20 0 1 0 338658823 289886208 6038 33554432000 4194304 5683644 140735751876304 140735751867392 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/7458/statm: 70773 6038 369 364 0 5996 0
[pid=7460] ppid=7458 vsize=143896 CPUtime=23.49 cores=0,2,4,6
/proc/7460/stat : 7460 (scip-2.0.1.linu) R 7458 7460 7460 0 -1 4202496 39220 0 0 0 2312 37 0 0 20 0 1 0 338658823 147349504 31954 33554432000 4194304 15243407 140735760118528 140735760112920 8137672 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/7460/statm: 35974 31954 949 2698 0 33269 0
[pid=7452/tid=7459] ppid=7450 vsize=358872 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7459/stat : 7459 (python) S 7450 7452 3621 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 2 0 338658823 367484928 7575 33554432000 4194304 5683644 140735751876304 139881587155504 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 24.11
Current children cumulated vsize (KiB) 785860

[startup+51.1003 s]
/proc/loadavg: 1.99 1.99 2.07 3/192 7462
/proc/meminfo: memFree=25981308/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=358872 CPUtime=0.59 cores=0,2,4,6
/proc/7452/stat : 7452 (python) S 7450 7452 3621 0 -1 4202496 10894 1011 1 0 47 12 0 0 20 0 2 0 338658624 367484928 7575 33554432000 4194304 5683644 140735751876304 140735751871136 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 54 0 0
/proc/7452/statm: 89718 7575 1900 364 0 24941 0
[pid=7458] ppid=7452 vsize=283092 CPUtime=0.07 cores=0,2,4,6
/proc/7458/stat : 7458 (python) S 7452 7452 3621 0 -1 4202560 1014 0 0 0 4 3 0 0 20 0 1 0 338658823 289886208 6038 33554432000 4194304 5683644 140735751876304 140735751867392 213419655592 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/7458/statm: 70773 6038 369 364 0 5996 0
[pid=7460] ppid=7458 vsize=144832 CPUtime=49.09 cores=0,2,4,6
/proc/7460/stat : 7460 (scip-2.0.1.linu) R 7458 7460 7460 0 -1 4202496 45117 0 0 0 4868 41 0 0 20 0 1 0 338658823 148307968 32073 33554432000 4194304 15243407 140735760118528 140735760112920 8112819 0 0 16781312 2 0 0 0 17 2 0 0 0 0 0
/proc/7460/statm: 36208 32073 949 2698 0 33503 0
[pid=7452/tid=7459] ppid=7450 vsize=358872 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7459/stat : 7459 (python) S 7450 7452 3621 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 2 0 338658823 367484928 7575 33554432000 4194304 5683644 140735751876304 139881587155504 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 49.75
Current children cumulated vsize (KiB) 786796

[startup+102.301 s]
/proc/loadavg: 2.05 2.00 2.07 3/222 7494
/proc/meminfo: memFree=23165992/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=510960 CPUtime=0.64 cores=0,2,4,6
/proc/7452/stat : 7452 (python) S 7450 7452 3621 0 -1 4202496 12720 1011 1 0 51 13 0 0 20 0 4 0 338658624 523223040 7718 33554432000 4194304 5683644 140735751876304 140735751871136 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 54 0 0
/proc/7452/statm: 127740 7718 1905 364 0 62957 0
[pid=7458] ppid=7452 vsize=358872 CPUtime=0.07 cores=0,2,4,6
/proc/7458/stat : 7458 (python) S 7452 7452 3621 0 -1 4202560 1147 0 0 0 4 3 0 0 20 0 2 0 338658823 367484928 6073 33554432000 4194304 5683644 140735751876304 140735751867952 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7458/statm: 89718 6073 397 364 0 24941 0
[pid=7460] ppid=7458 vsize=144832 CPUtime=49.8 cores=0,2,4,6
/proc/7460/stat : 7460 (scip-2.0.1.linu) T 7458 7460 7460 0 -1 4202496 45117 0 0 0 4939 41 0 0 20 0 1 0 338658823 148307968 32073 33554432000 4194304 15243407 140735760118528 140735760112920 6951759 0 0 16781312 2 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/7460/statm: 36208 32073 949 2698 0 33503 0
[pid=7458/tid=7463] ppid=7452 vsize=358872 CPUtime=0 cores=0,2,4,6
/proc/7458/task/7463/stat : 7463 (python) S 7452 7452 3621 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338663806 367484928 6073 33554432000 4194304 5683644 140735751876304 139881587155504 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7464] ppid=7452 vsize=359388 CPUtime=0.06 cores=0,2,4,6
/proc/7464/stat : 7464 (python) S 7452 7452 3621 0 -1 4202560 1073 0 0 0 3 3 0 0 20 0 2 0 338663808 368013312 6186 33554432000 4194304 5683644 140735751876304 140735751867952 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7464/statm: 89847 6186 384 364 0 25067 0
[pid=7466] ppid=7464 vsize=9736 CPUtime=47.11 cores=0,2,4,6
/proc/7466/stat : 7466 (clasp-1.3.7-x86) T 7464 7466 7466 0 -1 4202496 2465 0 0 0 4711 0 0 0 20 0 1 0 338663809 9969664 2188 33554432000 134512640 136139019 4292469376 4292468352 134876667 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7466/statm: 2434 2188 182 398 0 2034 0
[pid=7464/tid=7491] ppid=7452 vsize=359388 CPUtime=0 cores=0,2,4,6
/proc/7464/task/7491/stat : 7491 (python) S 7452 7452 3621 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 338668521 368013312 6186 33554432000 4194304 5683644 140735751876304 139881587155504 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7492] ppid=7452 vsize=435180 CPUtime=0 cores=0,2,4,6
/proc/7492/stat : 7492 (python) S 7452 7452 3621 0 -1 4202560 808 0 0 0 0 0 0 0 20 0 1 0 338668524 445624320 6143 33554432000 4194304 5683644 140735751876304 140735751867392 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/7492/statm: 108795 6143 336 364 0 44012 0
[pid=7494] ppid=7492 vsize=98476 CPUtime=3.29 cores=0,2,4,6
/proc/7494/stat : 7494 (scip-2.0.1.linu) R 7492 7494 7494 0 -1 4202496 22490 0 0 0 320 9 0 0 20 0 1 0 338668524 100839424 21876 33554432000 4194304 13684479 140736067557888 140736067554200 11876362 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/7494/statm: 24619 21876 646 2317 0 22297 0
[pid=7452/tid=7459] ppid=7450 vsize=510960 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7459/stat : 7459 (python) S 7450 7452 3621 0 -1 4202560 26 1011 0 0 0 0 0 0 20 0 4 0 338658823 523223040 7718 33554432000 4194304 5683644 140735751876304 139881587155504 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7452/tid=7465] ppid=7450 vsize=510960 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7465/stat : 7465 (python) S 7450 7452 3621 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 4 0 338663809 523223040 7718 33554432000 4194304 5683644 140735751876304 139881576665648 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7452/tid=7493] ppid=7450 vsize=510960 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7493/stat : 7493 (python) S 7450 7452 3621 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 4 0 338668524 523223040 7718 33554432000 4194304 5683644 140735751876304 139881566175792 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 100.97
Current children cumulated vsize (KiB) 1917444

[startup+162.301 s]
/proc/loadavg: 2.02 2.00 2.06 3/192 7564
/proc/meminfo: memFree=26832312/32873844 swapFree=6608/6608

################
# More data... #
################

[pid=7452/tid=7465] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7465/stat : 7465 (python) S 7450 7452 3621 0 -1 4202560 30 1011 0 0 0 0 0 0 20 0 10 0 338663809 988889088 8278 33554432000 4194304 5683644 140735751876304 139881576665648 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7452/tid=7493] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7493/stat : 7493 (python) S 7450 7452 3621 0 -1 4202560 30 1011 0 0 0 0 0 0 20 0 10 0 338668524 988889088 8278 33554432000 4194304 5683644 140735751876304 139881566175792 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7452/tid=7566] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7566/stat : 7566 (python) S 7450 7452 3621 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 10 0 338677897 988889088 8278 33554432000 4194304 5683644 140735751876304 139881555685936 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7452/tid=7620] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7620/stat : 7620 (python) S 7450 7452 3621 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 10 0 338682626 988889088 8278 33554432000 4194304 5683644 140735751876304 139881545196080 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7452/tid=7654] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7654/stat : 7654 (python) S 7450 7452 3621 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 10 0 338692262 988889088 8278 33554432000 4194304 5683644 140735751876304 139881534706224 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=7452/tid=7677] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7677/stat : 7677 (python) S 7450 7452 3621 0 -1 4202560 20 1011 0 0 0 0 0 0 20 0 10 0 338715267 988889088 8278 33554432000 4194304 5683644 140735751876304 139881179243056 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7452/tid=7686] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7686/stat : 7686 (python) S 7450 7452 3621 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 10 0 338761388 988889088 8278 33554432000 4194304 5683644 140735751876304 139881168753200 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=7452/tid=7690] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7690/stat : 7690 (python) S 7450 7452 3621 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 10 0 338766153 988889088 8278 33554432000 4194304 5683644 140735751876304 139881158263344 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1078.27
Current children cumulated vsize (KiB) 16603724

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1075.9 s]
/proc/loadavg: 1.00 1.06 1.42 2/224 7691
/proc/meminfo: memFree=25047300/32873844 swapFree=6608/6608
[pid=7452] ppid=7450 vsize=965712 CPUtime=1.21 cores=0,2,4,6
/proc/7452/stat : 7452 (python) S 7450 7452 3621 0 -1 4202496 18732 1011 1 0 106 15 0 0 20 0 10 0 338658624 988889088 8278 33554432000 4194304 5683644 140735751876304 140735751871136 213423023373 0 0 16781312 33554434 18446744073709551615 0 0 17 4 0 0 54 0 0
/proc/7452/statm: 241428 8278 1917 364 0 176627 0
[pid=7458] ppid=7452 vsize=358872 CPUtime=0.15 cores=0,2,4,6
/proc/7458/stat : 7458 (python) S 7452 7452 3621 0 -1 4202560 1177 0 0 0 9 6 0 0 20 0 2 0 338658823 367484928 6074 33554432000 4194304 5683644 140735751876304 140735751867952 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7458/statm: 89718 6074 398 364 0 24941 0
[pid=7460] ppid=7458 vsize=140212 CPUtime=94.92 cores=0,2,4,6
/proc/7460/stat : 7460 (scip-2.0.1.linu) T 7458 7460 7460 0 -1 4202496 53202 0 0 0 9444 48 0 0 20 0 1 0 338658823 143577088 32476 33554432000 4194304 15243407 140735760118528 140735760113240 8113862 0 0 16781312 2 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/7460/statm: 35053 32476 949 2698 0 32348 0
[pid=7458/tid=7463] ppid=7452 vsize=358872 CPUtime=0 cores=0,2,4,6
/proc/7458/task/7463/stat : 7463 (python) S 7452 7452 3621 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338663806 367484928 6074 33554432000 4194304 5683644 140735751876304 139881587155504 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7464] ppid=7452 vsize=359388 CPUtime=0.38 cores=0,2,4,6
/proc/7464/stat : 7464 (python) S 7452 7452 3621 0 -1 4202560 1162 0 0 0 22 16 0 0 20 0 2 0 338663808 368013312 6186 33554432000 4194304 5683644 140735751876304 140735751867952 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7464/statm: 89847 6186 384 364 0 25067 0
[pid=7466] ppid=7464 vsize=14920 CPUtime=229.59 cores=0,2,4,6
/proc/7466/stat : 7466 (clasp-1.3.7-x86) T 7464 7466 7466 0 -1 4202496 3751 0 0 0 22958 1 0 0 20 0 1 0 338663809 15278080 3474 33554432000 134512640 136139019 4292469376 4292468256 134742409 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7466/statm: 3730 3474 182 398 0 3330 0
[pid=7464/tid=7491] ppid=7452 vsize=359388 CPUtime=0 cores=0,2,4,6
/proc/7464/task/7491/stat : 7491 (python) S 7452 7452 3621 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 338668521 368013312 6186 33554432000 4194304 5683644 140735751876304 139881587155504 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7492] ppid=7452 vsize=435180 CPUtime=0.91 cores=0,2,4,6
/proc/7492/stat : 7492 (python) S 7452 7452 3621 0 -1 4202560 1189 0 0 0 54 37 0 0 20 0 2 0 338668524 445624320 6192 33554432000 4194304 5683644 140735751876304 140735751867952 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7492/statm: 108795 6192 384 364 0 44012 0
[pid=7494] ppid=7492 vsize=161672 CPUtime=506.81 cores=0,2,4,6
/proc/7494/stat : 7494 (scip-2.0.1.linu) T 7492 7494 7494 0 -1 4202496 39575 0 0 0 50439 242 0 0 20 0 1 0 338668524 165552128 38268 33554432000 4194304 13684479 140736067557888 140736067551720 7390783 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7494/statm: 40418 38268 815 2317 0 38096 0
[pid=7492/tid=7496] ppid=7452 vsize=435180 CPUtime=0 cores=0,2,4,6
/proc/7492/task/7496/stat : 7496 (python) S 7452 7452 3621 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 2 0 338673347 445624320 6192 33554432000 4194304 5683644 140735751876304 139881576665648 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7565] ppid=7452 vsize=510972 CPUtime=0.07 cores=0,2,4,6
/proc/7565/stat : 7565 (python) S 7452 7452 3621 0 -1 4202560 1127 0 0 0 4 3 0 0 20 0 2 0 338677897 523235328 6198 33554432000 4194304 5683644 140735751876304 140735751867952 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7565/statm: 127743 6198 384 364 0 62957 0
[pid=7567] ppid=7565 vsize=79748 CPUtime=47.24 cores=0,2,4,6
/proc/7567/stat : 7567 (minion) T 7565 7567 7567 0 -1 4202496 5985 0 0 0 4721 3 0 0 20 0 1 0 338677897 81661952 12644 33554432000 134512640 165565024 4288395552 4288392864 158624745 0 0 16781312 8396802 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/7567/statm: 19937 12644 336 7582 0 12352 0
[pid=7565/tid=7618] ppid=7452 vsize=510972 CPUtime=0 cores=0,2,4,6
/proc/7565/task/7618/stat : 7618 (python) S 7452 7452 3621 0 -1 4202560 44 0 0 0 0 0 0 0 20 0 2 0 338682623 523235328 6198 33554432000 4194304 5683644 140735751876304 139881566175792 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=7619] ppid=7452 vsize=586764 CPUtime=0.07 cores=0,2,4,6
/proc/7619/stat : 7619 (python) S 7452 7452 3621 0 -1 4202560 1123 0 0 0 4 3 0 0 20 0 2 0 338682626 600846336 6204 33554432000 4194304 5683644 140735751876304 140735751867952 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7619/statm: 146691 6204 384 364 0 81902 0
[pid=7621] ppid=7619 vsize=47676 CPUtime=51.15 cores=0,2,4,6
/proc/7621/stat : 7621 (pbct-0.1.2-linu) T 7619 7621 7621 0 -1 4202496 12036 0 0 0 4669 446 0 0 20 0 1 0 338682626 48820224 11310 33554432000 134512640 136422188 4289232352 4289229804 135874785 0 0 16781312 16387 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7621/statm: 11919 11310 214 467 0 11450 0
[pid=7619/tid=7651] ppid=7452 vsize=586764 CPUtime=0 cores=0,2,4,6
/proc/7619/task/7651/stat : 7651 (python) S 7452 7452 3621 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338687744 600846336 6204 33554432000 4194304 5683644 140735751876304 139881555685936 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=7653] ppid=7452 vsize=662556 CPUtime=0.07 cores=0,2,4,6
/proc/7653/stat : 7653 (python) S 7452 7452 3621 0 -1 4202560 1140 0 0 0 5 2 0 0 20 0 2 0 338692262 678457344 6214 33554432000 4194304 5683644 140735751876304 140735751867952 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7653/statm: 165639 6214 384 364 0 100847 0
[pid=7655] ppid=7653 vsize=9624972 CPUtime=49.72 cores=0,2,4,6
/proc/7655/stat : 7655 (java) T 7653 7655 7655 0 -1 4202496 21948 0 1 0 4927 45 0 0 20 0 19 0 338692262 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140735707972720 213422997565 0 0 0 16800975 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7655/statm: 2406243 433816 2267 9 0 2396937 0
[pid=7655/tid=7656] ppid=7653 vsize=9624972 CPUtime=46.99 cores=0,2,4,6
/proc/7655/task/7656/stat : 7656 (java) T 7653 7655 7655 0 -1 4202560 3804 0 1 0 4671 28 0 0 20 0 19 0 338692264 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723603762288 140723476747742 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7655/tid=7657] ppid=7653 vsize=9624972 CPUtime=0.12 cores=0,2,4,6
/proc/7655/task/7657/stat : 7657 (java) T 7653 7655 7655 0 -1 4202560 253 0 0 0 10 2 0 0 20 0 19 0 338692265 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723447744680 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7655/tid=7658] ppid=7653 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/7655/task/7658/stat : 7658 (java) T 7653 7655 7655 0 -1 4202560 31 0 0 0 9 0 0 0 20 0 19 0 338692265 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723446692136 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7655/tid=7659] ppid=7653 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/7655/task/7659/stat : 7659 (java) T 7653 7655 7655 0 -1 4202560 172 0 0 0 7 2 0 0 20 0 19 0 338692265 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723445639592 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7655/tid=7660] ppid=7653 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/7655/task/7660/stat : 7660 (java) T 7653 7655 7655 0 -1 4202560 169 0 0 0 8 2 0 0 20 0 19 0 338692265 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723444587048 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7655/tid=7661] ppid=7653 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/7655/task/7661/stat : 7661 (java) T 7653 7655 7655 0 -1 4202560 34 0 0 0 9 1 0 0 20 0 19 0 338692265 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723443534504 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7655/tid=7662] ppid=7653 vsize=9624972 CPUtime=0.11 cores=0,2,4,6
/proc/7655/task/7662/stat : 7662 (java) T 7653 7655 7655 0 -1 4202560 69 0 0 0 10 1 0 0 20 0 19 0 338692265 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723442481960 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 6 0 0 0 0 0
[pid=7655/tid=7663] ppid=7653 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/7655/task/7663/stat : 7663 (java) T 7653 7655 7655 0 -1 4202560 93 0 0 0 8 2 0 0 20 0 19 0 338692265 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723441429416 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7655/tid=7664] ppid=7653 vsize=9624972 CPUtime=0.12 cores=0,2,4,6
/proc/7655/task/7664/stat : 7664 (java) T 7653 7655 7655 0 -1 4202560 48 0 0 0 10 2 0 0 20 0 19 0 338692265 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723440376872 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7655/tid=7665] ppid=7653 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7655/task/7665/stat : 7665 (java) T 7653 7655 7655 0 -1 4202560 741 0 0 0 0 0 0 0 20 0 19 0 338692266 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723427965032 213423011753 0 0 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7655/tid=7666] ppid=7653 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7655/task/7666/stat : 7666 (java) T 7653 7655 7655 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338692266 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723426911352 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7655/tid=7667] ppid=7653 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7655/task/7667/stat : 7667 (java) T 7653 7655 7655 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338692266 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723425858696 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7655/tid=7668] ppid=7653 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7655/task/7668/stat : 7668 (java) T 7653 7655 7655 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338692269 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723424807440 213423019808 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7655/tid=7669] ppid=7653 vsize=9624972 CPUtime=0.87 cores=0,2,4,6
/proc/7655/task/7669/stat : 7669 (java) T 7653 7655 7655 0 -1 4202560 6326 0 0 0 85 2 0 0 20 0 19 0 338692269 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723423754808 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7655/tid=7670] ppid=7653 vsize=9624972 CPUtime=0.74 cores=0,2,4,6
/proc/7655/task/7670/stat : 7670 (java) T 7653 7655 7655 0 -1 4202560 8784 0 0 0 72 2 0 0 20 0 19 0 338692269 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723422702264 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7655/tid=7671] ppid=7653 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7655/task/7671/stat : 7671 (java) T 7653 7655 7655 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338692269 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723421649912 213423010876 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7655/tid=7672] ppid=7653 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7655/task/7672/stat : 7672 (java) T 7653 7655 7655 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338692269 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723420597576 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7655/tid=7673] ppid=7653 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7655/task/7673/stat : 7673 (java) T 7653 7655 7655 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338692298 9855971328 433816 33554432000 1073741824 1073778376 140735707981584 140723419542312 213423011753 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7653/tid=7675] ppid=7452 vsize=662556 CPUtime=0 cores=0,2,4,6
/proc/7653/task/7675/stat : 7675 (python) S 7452 7452 3621 0 -1 4202560 44 0 0 0 0 0 0 0 20 0 2 0 338697009 678457344 6214 33554432000 4194304 5683644 140735751876304 139881545196080 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7676] ppid=7452 vsize=738348 CPUtime=0.08 cores=0,2,4,6
/proc/7676/stat : 7676 (python) S 7452 7452 3621 0 -1 4202560 1171 0 0 0 4 4 0 0 20 0 2 0 338715267 756068352 6220 33554432000 4194304 5683644 140735751876304 140735751867952 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7676/statm: 184587 6220 384 364 0 119792 0
[pid=7678] ppid=7676 vsize=106360 CPUtime=47.61 cores=0,2,4,6
/proc/7678/stat : 7678 (bsolo_pb10) T 7676 7678 7678 0 -1 4202496 24492 0 0 0 4749 12 0 0 20 0 1 0 338715268 108912640 24154 33554432000 134512640 137138936 4289951664 4289949976 136504438 0 0 16781312 16384 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7678/statm: 26590 24154 192 642 0 25946 0
[pid=7676/tid=7679] ppid=7452 vsize=738348 CPUtime=0 cores=0,2,4,6
/proc/7676/task/7679/stat : 7679 (python) S 7452 7452 3621 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 2 0 338720031 756068352 6220 33554432000 4194304 5683644 140735751876304 139881534706224 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7685] ppid=7452 vsize=814140 CPUtime=0.07 cores=0,2,4,6
/proc/7685/stat : 7685 (python) S 7452 7452 3621 0 -1 4202560 1207 0 0 0 4 3 0 0 20 0 2 0 338761388 833679360 6739 33554432000 4194304 5683644 140735751876304 140735751867952 213423023373 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7685/statm: 203535 6739 390 364 0 138737 0
[pid=7687] ppid=7685 vsize=93644 CPUtime=47.6 cores=0,2,4,6
/proc/7687/stat : 7687 (bsolo_pb10) T 7685 7687 7687 0 -1 4202496 21316 0 0 0 4755 5 0 0 20 0 1 0 338761389 95891456 20977 33554432000 134512640 137138936 4291557088 4291550768 134618214 0 0 16781312 16384 18446744071579387992 0 0 17 2 0 0 0 0 0
/proc/7687/statm: 23411 20977 192 642 0 22767 0
[pid=7685/tid=7688] ppid=7452 vsize=814140 CPUtime=0 cores=0,2,4,6
/proc/7685/task/7688/stat : 7688 (python) S 7452 7452 3621 0 -1 4202560 40 0 0 0 0 0 0 0 20 0 2 0 338766152 833679360 6739 33554432000 4194304 5683644 140735751876304 139881179243056 213423019808 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7689] ppid=7452 vsize=889932 CPUtime=0 cores=0,2,4,6
/proc/7689/stat : 7689 (python) S 7452 7452 3621 0 -1 4202560 891 0 0 0 0 0 0 0 20 0 1 0 338766153 911290368 6698 33554432000 4194304 5683644 140735751876304 140735751867392 213419655651 0 0 16781312 33554946 18446744071580425449 0 0 17 6 0 0 0 0 0
/proc/7689/statm: 222483 6698 343 364 0 157682 0
[pid=7691] ppid=7689 vsize=12656 CPUtime=0.62 cores=0,2,4,6
/proc/7691/stat : 7691 (bsolo_pb10) R 7689 7691 7691 0 -1 4202496 1072 0 0 0 61 1 0 0 20 0 1 0 338766153 12959744 730 33554432000 134512640 137138936 4293116672 4293115788 136570369 0 0 16781312 16384 0 0 0 17 2 0 0 0 0 0
/proc/7691/statm: 3164 730 191 642 0 2520 0
[pid=7452/tid=7459] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7459/stat : 7459 (python) S 7450 7452 3621 0 -1 4202560 31 1011 0 0 0 0 0 0 20 0 10 0 338658823 988889088 8278 33554432000 4194304 5683644 140735751876304 139881587155504 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=7452/tid=7465] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7465/stat : 7465 (python) S 7450 7452 3621 0 -1 4202560 30 1011 0 0 0 0 0 0 20 0 10 0 338663809 988889088 8278 33554432000 4194304 5683644 140735751876304 139881576665648 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7452/tid=7493] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7493/stat : 7493 (python) S 7450 7452 3621 0 -1 4202560 30 1011 0 0 0 0 0 0 20 0 10 0 338668524 988889088 8278 33554432000 4194304 5683644 140735751876304 139881566175792 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7452/tid=7566] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7566/stat : 7566 (python) S 7450 7452 3621 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 10 0 338677897 988889088 8278 33554432000 4194304 5683644 140735751876304 139881555685936 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7452/tid=7620] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7620/stat : 7620 (python) S 7450 7452 3621 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 10 0 338682626 988889088 8278 33554432000 4194304 5683644 140735751876304 139881545196080 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7452/tid=7654] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7654/stat : 7654 (python) S 7450 7452 3621 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 10 0 338692262 988889088 8278 33554432000 4194304 5683644 140735751876304 139881534706224 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=7452/tid=7677] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7677/stat : 7677 (python) S 7450 7452 3621 0 -1 4202560 20 1011 0 0 0 0 0 0 20 0 10 0 338715267 988889088 8278 33554432000 4194304 5683644 140735751876304 139881179243056 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7452/tid=7686] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7686/stat : 7686 (python) S 7450 7452 3621 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 10 0 338761388 988889088 8278 33554432000 4194304 5683644 140735751876304 139881168753200 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=7452/tid=7690] ppid=7450 vsize=965712 CPUtime=0 cores=0,2,4,6
/proc/7452/task/7690/stat : 7690 (python) S 7450 7452 3621 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 10 0 338766153 988889088 8278 33554432000 4194304 5683644 140735751876304 139881158263344 213423019808 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1078.27
Current children cumulated vsize (KiB) 16603724

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 7452 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=70837
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=156976
# CPU time returned by wait4() is 1.22781
# while last known CPU time is 1078.27
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1075.97
CPU time (s): 1078.27
CPU user time (s): 1069.35
CPU system time (s): 8.92
CPU usage (%): 100.214
Max. virtual memory (cumulated for all children) (KiB): 16603724

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.07084
system time used= 0.156976
maximum resident set size= 33220
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 20256
page faults= 1
swaps= 0
block input operations= 0
block output operations= 1064
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2082
involuntary context switches= 19

runsolver used 6.76197 second user time and 10.7124 second system time

The end

Launcher Data 

Begin job on node140 at 2012-06-03 14:08:56
IDJOB=3742655
IDBENCH=90220
IDSOLVER=2327
FILE ID=node140/3742655-1338725341
RUNJOBID= node140-1338713222-3637
PBS_JOBID= 14636725
Free space on /tmp= 70524 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathBA/normalized-BeauxArts_K66.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742655-1338725341/watcher-3742655-1338725341 -o /tmp/evaluation-result-3742655-1338725341/solver-3742655-1338725341 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742655-1338725341.opb -seed 1222798511 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 03e282ee85097a100b1dfa3533d9978b
RANDOM SEED=1222798511

node140.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        26334384 kB
Buffers:          489928 kB
Cached:          4126936 kB
SwapCached:         1620 kB
Active:          2649736 kB
Inactive:        2953304 kB
Active(anon):     977144 kB
Inactive(anon):    11328 kB
Active(file):    1672592 kB
Inactive(file):  2941976 kB
Unevictable:        6608 kB
Mlocked:            6608 kB
SwapTotal:      67108856 kB
SwapFree:       67105264 kB
Dirty:            101868 kB
Writeback:             0 kB
AnonPages:        991476 kB
Mapped:            22664 kB
Shmem:                56 kB
Slab:             792508 kB
SReclaimable:     233436 kB
SUnreclaim:       559072 kB
KernelStack:        1528 kB
PageTables:         8304 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:    1291856 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70524 MiB
End job on node140 at 2012-06-03 14:26:56