Trace number 3742343

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 283.81 281.555

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K129.opb
MD5SUM1a22ba01765c232fe0c8623cfb5d750f
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables15351
Total number of constraints15611
Number of constraints which are clauses15482
Number of constraints which are cardinality constraints (but not clauses)129
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint129
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 130
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.35/1.17	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.35/1.18	c INFO: solving HOME/instance-3742343-1338724530.opb
0.52/1.34	c INFO: parsing took 0.16 s
0.82/1.66	c INFO: features took 0.31 CPU seconds
0.82/1.69	c INFO: running scip-2.0.1-clp@0 for 45 with 1799 remaining (b = 0.61)
50.56/51.34	c INFO: running clasp-1.3.7@0 for 45 with 1750 remaining (b = 0.43)
97.69/98.49	c INFO: running scip-2.0.1-spx@0 for 45 with 1702 remaining (b = 0.05)
148.35/149.07	c INFO: running scip-2.0.1-clp@49 for 45 with 1652 remaining (b = 0.01)
191.82/192.45	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1609 remaining (b = 0.04)
238.20/236.14	c INFO: running clasp-1.3.7@47 for 45 with 1563 remaining (b = 0.01)
283.37/281.28	c INFO: running sat4j-pb-v20101225@0 for 45 with 1518 remaining (b = 0.14)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742343-1338724530/watcher-3742343-1338724530 -o /tmp/evaluation-result-3742343-1338724530/solver-3742343-1338724530 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742343-1338724530.opb -seed 639862488 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.99 2.00 2.04 3/183 9346
/proc/meminfo: memFree=30860760/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=14840 CPUtime=0 cores=1,3,5,7
/proc/9346/stat : 9346 (python) R 9344 9346 3958 0 -1 4202496 391 0 0 0 0 0 0 0 20 0 1 0 338579160 15196160 306 33554432000 4194304 5683644 140737024878048 140735728808200 4540219 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 3710 308 217 364 0 152 0

[startup+0.0446451 s]
/proc/loadavg: 1.99 2.00 2.04 3/183 9346
/proc/meminfo: memFree=30860760/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=16104 CPUtime=0 cores=1,3,5,7
/proc/9346/stat : 9346 (python) D 9344 9346 3958 0 -1 4202496 1014 0 0 0 0 0 0 0 20 0 1 0 338579160 16490496 846 33554432000 4194304 5683644 140737024878048 140737024807048 217410519952 0 0 16781312 2 18446744072100588836 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 4026 846 368 364 0 468 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 16104

[startup+0.10019 s]
/proc/loadavg: 1.99 2.00 2.04 3/183 9346
/proc/meminfo: memFree=30860760/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=19084 CPUtime=0.02 cores=1,3,5,7
/proc/9346/stat : 9346 (python) R 9344 9346 3958 0 -1 4202496 1398 0 0 0 2 0 0 0 20 0 1 0 338579160 19542016 1153 33554432000 4194304 5683644 140737024878048 140737024812984 4865408 0 0 16781312 2 0 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 4771 1153 430 364 0 695 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 19084

[startup+0.300263 s]
/proc/loadavg: 1.99 2.00 2.04 3/183 9346
/proc/meminfo: memFree=30860760/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=102076 CPUtime=0.11 cores=1,3,5,7
/proc/9346/stat : 9346 (python) D 9344 9346 3958 0 -1 4202496 4201 1010 1 0 9 2 0 0 20 0 1 0 338579160 104525824 2754 33554432000 4194304 5683644 140737024878048 140737024746408 217410519952 0 0 16781312 2 0 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 25519 2754 890 364 0 1729 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 102076

[startup+0.700405 s]
/proc/loadavg: 1.99 2.00 2.04 3/183 9346
/proc/meminfo: memFree=30860760/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=166944 CPUtime=0.22 cores=1,3,5,7
/proc/9346/stat : 9346 (python) R 9344 9346 3958 0 -1 4202496 6561 1010 1 0 17 5 0 0 20 0 1 0 338579160 170950656 4351 33554432000 4194304 5683644 140737024878048 140737024782280 5291431 0 0 16781312 2 0 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 41736 4351 1227 364 0 3289 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 166944

[startup+1.50021 s]
/proc/loadavg: 2.07 2.01 2.05 2/184 9351
/proc/meminfo: memFree=30847832/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=287956 CPUtime=0.72 cores=1,3,5,7
/proc/9346/stat : 9346 (python) R 9344 9346 3958 0 -1 4202496 12743 1010 1 0 63 9 0 0 20 0 1 0 338579160 294866944 8738 33554432000 4194304 5683644 140737024878048 140737024792120 139681837076501 0 0 16781312 33554434 0 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 71989 8738 1846 364 0 7218 0
Current children cumulated CPU time (s) 0.72
Current children cumulated vsize (KiB) 287956

[startup+3.10035 s]
/proc/loadavg: 2.07 2.01 2.05 3/187 9354
/proc/meminfo: memFree=30767604/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=364004 CPUtime=0.91 cores=1,3,5,7
/proc/9346/stat : 9346 (python) S 9344 9346 3958 0 -1 4202496 13123 1010 1 0 81 10 0 0 20 0 2 0 338579160 372740096 8894 33554432000 4194304 5683644 140737024878048 140737024872880 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 91001 8894 1902 364 0 26224 0
[pid=9352] ppid=9346 vsize=288224 CPUtime=0 cores=1,3,5,7
/proc/9352/stat : 9352 (python) S 9346 9346 3958 0 -1 4202560 766 0 0 0 0 0 0 0 20 0 1 0 338579329 295141376 7326 33554432000 4194304 5683644 140737024878048 140737024869136 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 5 0 0 0 0 0
/proc/9352/statm: 72056 7326 340 364 0 7279 0
[pid=9354] ppid=9352 vsize=153476 CPUtime=1.4 cores=1,3,5,7
/proc/9354/stat : 9354 (scip-2.0.1.linu) R 9352 9354 9354 0 -1 4202496 35105 0 0 0 131 9 0 0 20 0 1 0 338579329 157159424 34393 33554432000 4194304 15243407 140735090896704 140735090892376 4880933 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/9354/statm: 38369 34393 682 2698 0 35664 0
[pid=9346/tid=9353] ppid=9344 vsize=364004 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9353/stat : 9353 (python) S 9344 9346 3958 0 -1 4202560 26 1010 0 0 0 0 0 0 20 0 2 0 338579329 372740096 8894 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 2.31
Current children cumulated vsize (KiB) 805704
heavy processes:

[startup+6.30032 s]
/proc/loadavg: 2.07 2.01 2.05 3/187 9354
/proc/meminfo: memFree=30657060/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=364004 CPUtime=0.91 cores=1,3,5,7
/proc/9346/stat : 9346 (python) S 9344 9346 3958 0 -1 4202496 13123 1010 1 0 81 10 0 0 20 0 2 0 338579160 372740096 8894 33554432000 4194304 5683644 140737024878048 140737024872880 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 91001 8894 1902 364 0 26224 0
[pid=9352] ppid=9346 vsize=288224 CPUtime=0 cores=1,3,5,7
/proc/9352/stat : 9352 (python) S 9346 9346 3958 0 -1 4202560 889 0 0 0 0 0 0 0 20 0 1 0 338579329 295141376 7349 33554432000 4194304 5683644 140737024878048 140737024869136 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/9352/statm: 72056 7349 363 364 0 7279 0
[pid=9354] ppid=9352 vsize=207036 CPUtime=4.59 cores=1,3,5,7
/proc/9354/stat : 9354 (scip-2.0.1.linu) R 9352 9354 9354 0 -1 4202496 47221 0 0 0 449 10 0 0 20 0 1 0 338579329 212004864 46509 33554432000 4194304 15243407 140735090896704 140735090892088 4880912 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/9354/statm: 51759 46509 682 2698 0 49054 0
[pid=9346/tid=9353] ppid=9344 vsize=364004 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9353/stat : 9353 (python) S 9344 9346 3958 0 -1 4202560 26 1010 0 0 0 0 0 0 20 0 2 0 338579329 372740096 8894 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 5.5
Current children cumulated vsize (KiB) 859264

[startup+12.7004 s]
/proc/loadavg: 2.06 2.01 2.05 3/187 9354
/proc/meminfo: memFree=30595528/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=364004 CPUtime=0.91 cores=1,3,5,7
/proc/9346/stat : 9346 (python) S 9344 9346 3958 0 -1 4202496 13123 1010 1 0 81 10 0 0 20 0 2 0 338579160 372740096 8894 33554432000 4194304 5683644 140737024878048 140737024872880 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 91001 8894 1902 364 0 26224 0
[pid=9352] ppid=9346 vsize=288224 CPUtime=0 cores=1,3,5,7
/proc/9352/stat : 9352 (python) S 9346 9346 3958 0 -1 4202560 896 0 0 0 0 0 0 0 20 0 1 0 338579329 295141376 7349 33554432000 4194304 5683644 140737024878048 140737024869136 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/9352/statm: 72056 7349 363 364 0 7279 0
[pid=9354] ppid=9352 vsize=274244 CPUtime=11 cores=1,3,5,7
/proc/9354/stat : 9354 (scip-2.0.1.linu) R 9352 9354 9354 0 -1 4202496 63420 0 0 0 1085 15 0 0 20 0 1 0 338579329 280825856 62708 33554432000 4194304 15243407 140735090896704 140735090893016 13277498 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0
/proc/9354/statm: 68561 62708 682 2698 0 65856 0
[pid=9346/tid=9353] ppid=9344 vsize=364004 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9353/stat : 9353 (python) S 9344 9346 3958 0 -1 4202560 26 1010 0 0 0 0 0 0 20 0 2 0 338579329 372740096 8894 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 11.91
Current children cumulated vsize (KiB) 926472

[startup+25.5003 s]
/proc/loadavg: 2.05 2.01 2.04 3/191 9358
/proc/meminfo: memFree=30263248/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=364004 CPUtime=0.91 cores=1,3,5,7
/proc/9346/stat : 9346 (python) S 9344 9346 3958 0 -1 4202496 13123 1010 1 0 81 10 0 0 20 0 2 0 338579160 372740096 8894 33554432000 4194304 5683644 140737024878048 140737024872880 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 91001 8894 1902 364 0 26224 0
[pid=9352] ppid=9346 vsize=288224 CPUtime=0.03 cores=1,3,5,7
/proc/9352/stat : 9352 (python) S 9346 9346 3958 0 -1 4202560 1000 0 0 0 2 1 0 0 20 0 1 0 338579329 295141376 7354 33554432000 4194304 5683644 140737024878048 140737024869136 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 1 0 0 0 0 0
/proc/9352/statm: 72056 7354 368 364 0 7279 0
[pid=9354] ppid=9352 vsize=388588 CPUtime=23.79 cores=1,3,5,7
/proc/9354/stat : 9354 (scip-2.0.1.linu) R 9352 9354 9354 0 -1 4202496 91774 0 0 0 2352 27 0 0 20 0 1 0 338579329 397914112 91062 33554432000 4194304 15243407 140735090896704 140735090893016 4865584 0 0 16781312 2 0 0 0 17 5 0 0 0 0 0
/proc/9354/statm: 97147 91062 686 2698 0 94442 0
[pid=9346/tid=9353] ppid=9344 vsize=364004 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9353/stat : 9353 (python) S 9344 9346 3958 0 -1 4202560 26 1010 0 0 0 0 0 0 20 0 2 0 338579329 372740096 8894 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 24.73
Current children cumulated vsize (KiB) 1040816

[startup+51.1004 s]
/proc/loadavg: 2.03 2.01 2.04 3/191 9358
/proc/meminfo: memFree=29897344/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=364004 CPUtime=0.91 cores=1,3,5,7
/proc/9346/stat : 9346 (python) S 9344 9346 3958 0 -1 4202496 13123 1010 1 0 81 10 0 0 20 0 2 0 338579160 372740096 8894 33554432000 4194304 5683644 140737024878048 140737024872880 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 91001 8894 1902 364 0 26224 0
[pid=9352] ppid=9346 vsize=288224 CPUtime=0.06 cores=1,3,5,7
/proc/9352/stat : 9352 (python) S 9346 9346 3958 0 -1 4202560 1018 0 0 0 3 3 0 0 20 0 1 0 338579329 295141376 7354 33554432000 4194304 5683644 140737024878048 140737024869136 217410535848 0 0 16781312 33554946 18446744071580425449 0 0 17 1 0 0 0 0 0
/proc/9352/statm: 72056 7354 368 364 0 7279 0
[pid=9354] ppid=9352 vsize=542876 CPUtime=49.39 cores=1,3,5,7
/proc/9354/stat : 9354 (scip-2.0.1.linu) R 9352 9354 9354 0 -1 4202496 130711 0 0 0 4893 46 0 0 20 0 1 0 338579329 555905024 129999 33554432000 4194304 15243407 140735090896704 140735090893000 4880915 0 0 16781312 2 0 0 0 17 5 0 0 0 0 0
/proc/9354/statm: 135719 129999 686 2698 0 133014 0
[pid=9346/tid=9353] ppid=9344 vsize=364004 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9353/stat : 9353 (python) S 9344 9346 3958 0 -1 4202560 26 1010 0 0 0 0 0 0 20 0 2 0 338579329 372740096 8894 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 50.36
Current children cumulated vsize (KiB) 1195104

[startup+102.307 s]
/proc/loadavg: 2.01 2.00 2.04 3/200 9367
/proc/meminfo: memFree=29361144/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=515588 CPUtime=0.95 cores=1,3,5,7
/proc/9346/stat : 9346 (python) S 9344 9346 3958 0 -1 4202496 14961 1010 1 0 85 10 0 0 20 0 4 0 338579160 527962112 9419 33554432000 4194304 5683644 140737024878048 140737024872880 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9346/statm: 128897 9419 1907 364 0 64114 0
[pid=9352] ppid=9346 vsize=364004 CPUtime=0.07 cores=1,3,5,7
/proc/9352/stat : 9352 (python) S 9346 9346 3958 0 -1 4202560 1154 0 0 0 4 3 0 0 20 0 2 0 338579329 372740096 7389 33554432000 4194304 5683644 140737024878048 140737024869696 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9352/statm: 91001 7389 396 364 0 26224 0
[pid=9354] ppid=9352 vsize=544908 CPUtime=49.6 cores=1,3,5,7
/proc/9354/stat : 9354 (scip-2.0.1.linu) T 9352 9354 9354 0 -1 4202496 131106 0 0 0 4914 46 0 0 20 0 1 0 338579329 557985792 130394 33554432000 4194304 15243407 140735090896704 140735090893016 4880960 0 0 16781312 2 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/9354/statm: 136227 130394 686 2698 0 133522 0
[pid=9352/tid=9359] ppid=9346 vsize=364004 CPUtime=0 cores=1,3,5,7
/proc/9352/task/9359/stat : 9359 (python) S 9346 9346 3958 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 338584291 372740096 7389 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9360] ppid=9346 vsize=364016 CPUtime=0.07 cores=1,3,5,7
/proc/9360/stat : 9360 (python) S 9346 9346 3958 0 -1 4202560 1087 0 0 0 4 3 0 0 20 0 2 0 338584294 372752384 7376 33554432000 4194304 5683644 140737024878048 140737024869696 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9360/statm: 91004 7376 383 364 0 26224 0
[pid=9362] ppid=9360 vsize=21932 CPUtime=47.1 cores=1,3,5,7
/proc/9362/stat : 9362 (clasp-1.3.7-x86) T 9360 9362 9362 0 -1 4202496 5727 0 0 0 4709 1 0 0 20 0 1 0 338584294 22458368 5239 33554432000 134512640 136139019 4287678240 4287677216 134634749 0 0 16781312 16386 18446744071579387992 0 0 17 1 0 0 0 0 0
/proc/9362/statm: 5483 5239 182 398 0 5083 0
[pid=9360/tid=9364] ppid=9346 vsize=364016 CPUtime=0 cores=1,3,5,7
/proc/9360/task/9364/stat : 9364 (python) S 9346 9346 3958 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 338589007 372752384 7376 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9365] ppid=9346 vsize=439808 CPUtime=0 cores=1,3,5,7
/proc/9365/stat : 9365 (python) S 9346 9346 3958 0 -1 4202560 922 0 0 0 0 0 0 0 20 0 1 0 338589009 450363392 7356 33554432000 4194304 5683644 140737024878048 140737024869136 217410535907 0 0 16781312 33554946 18446744071580425449 0 0 17 1 0 0 0 0 0
/proc/9365/statm: 109952 7356 358 364 0 45169 0
[pid=9367] ppid=9365 vsize=200140 CPUtime=3.8 cores=1,3,5,7
/proc/9367/stat : 9367 (scip-2.0.1.linu) R 9365 9367 9367 0 -1 4202496 45559 0 0 0 370 10 0 0 20 0 1 0 338589009 204943360 44834 33554432000 4194304 13684479 140736677566768 140736677562152 4880912 0 0 16781312 2 0 0 0 17 5 0 0 0 0 0
/proc/9367/statm: 50035 44834 635 2317 0 47713 0
[pid=9346/tid=9353] ppid=9344 vsize=515588 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9353/stat : 9353 (python) S 9344 9346 3958 0 -1 4202560 26 1010 0 0 0 0 0 0 20 0 4 0 338579329 527962112 9419 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9346/tid=9361] ppid=9344 vsize=515588 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9361/stat : 9361 (python) S 9344 9346 3958 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 4 0 338584294 527962112 9419 33554432000 4194304 5683644 140737024878048 139681815675440 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9346/tid=9366] ppid=9344 vsize=515588 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9366/stat : 9366 (python) S 9344 9346 3958 0 -1 4202560 19 1010 0 0 0 0 0 0 20 0 4 0 338589009 527962112 9419 33554432000 4194304 5683644 140737024878048 139681731699248 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 101.59
Current children cumulated vsize (KiB) 2450396

[startup+162.301 s]
/proc/loadavg: 2.08 2.03 2.04 3/222 9390
/proc/meminfo: memFree=27149672/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=515588 CPUtime=0.98 cores=1,3,5,7

################
# More data... #
################

[pid=9492/tid=9504] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9504/stat : 9504 (java) S 9490 9492 9492 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 18 0 338607292 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265906992136 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9492/tid=9505] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9505/stat : 9505 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607295 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265905940624 217413900064 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9492/tid=9506] ppid=9490 vsize=9558408 CPUtime=0.07 cores=1,3,5,7
/proc/9492/task/9506/stat : 9506 (java) R 9490 9492 9492 0 -1 4202560 873 0 0 0 7 0 0 0 20 0 18 0 338607295 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265904876040 140266557683696 0 4 0 16800975 0 0 0 -1 3 0 0 0 0 0
[pid=9492/tid=9507] ppid=9490 vsize=9558408 CPUtime=0.06 cores=1,3,5,7
/proc/9492/task/9507/stat : 9507 (java) R 9490 9492 9492 0 -1 4202560 2216 0 0 0 6 0 0 0 20 0 18 0 338607295 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265903823240 140266559681448 0 4 0 16800975 0 0 0 -1 1 0 0 0 0 0
[pid=9492/tid=9508] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9508/stat : 9508 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607295 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265902783352 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=9492/tid=9509] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9509/stat : 9509 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607295 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265901730760 217413892009 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=9346/tid=9353] ppid=9344 vsize=667172 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9353/stat : 9353 (python) S 9344 9346 3958 0 -1 4202560 30 1010 0 0 0 0 0 0 20 0 6 0 338579329 683184128 9435 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9346/tid=9361] ppid=9344 vsize=667172 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9361/stat : 9361 (python) S 9344 9346 3958 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 6 0 338584294 683184128 9435 33554432000 4194304 5683644 140737024878048 139681815675440 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9346/tid=9366] ppid=9344 vsize=667172 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9366/stat : 9366 (python) S 9344 9346 3958 0 -1 4202560 19 1010 0 0 0 0 0 0 20 0 6 0 338589009 683184128 9435 33554432000 4194304 5683644 140737024878048 139681731699248 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9346/tid=9393] ppid=9344 vsize=667172 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9393/stat : 9393 (python) S 9344 9346 3958 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 6 0 338598405 683184128 9435 33554432000 4194304 5683644 140737024878048 139681721209392 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9346/tid=9491] ppid=9344 vsize=667172 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9491/stat : 9491 (python) S 9344 9346 3958 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 6 0 338607288 683184128 9435 33554432000 4194304 5683644 140737024878048 139681710719536 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 283.81
Current children cumulated vsize (KiB) 23483600

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+281.501 s]
/proc/loadavg: 1.98 2.00 2.02 5/239 9509
/proc/meminfo: memFree=28321344/32873648 swapFree=7628/7628
[pid=9346] ppid=9344 vsize=667172 CPUtime=1.06 cores=1,3,5,7
/proc/9346/stat : 9346 (python) S 9344 9346 3958 0 -1 4202496 16857 1010 1 0 95 11 0 0 20 0 6 0 338579160 683184128 9435 33554432000 4194304 5683644 140737024878048 140737024872880 217413903629 0 0 16781312 33554434 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9346/statm: 166793 9435 1911 364 0 102004 0
[pid=9352] ppid=9346 vsize=364004 CPUtime=0.14 cores=1,3,5,7
/proc/9352/stat : 9352 (python) S 9346 9346 3958 0 -1 4202560 1188 0 0 0 8 6 0 0 20 0 2 0 338579329 372740096 7390 33554432000 4194304 5683644 140737024878048 140737024869696 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9352/statm: 91001 7390 397 364 0 26224 0
[pid=9354] ppid=9352 vsize=774640 CPUtime=92.95 cores=1,3,5,7
/proc/9354/stat : 9354 (scip-2.0.1.linu) T 9352 9354 9354 0 -1 4202496 188279 0 0 0 9223 72 0 0 20 0 1 0 338579329 793231360 187900 33554432000 4194304 15243407 140735090896704 140735090893000 4865666 0 0 16781312 2 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/9354/statm: 193660 187900 686 2698 0 190955 0
[pid=9352/tid=9359] ppid=9346 vsize=364004 CPUtime=0 cores=1,3,5,7
/proc/9352/task/9359/stat : 9359 (python) S 9346 9346 3958 0 -1 4202560 49 0 0 0 0 0 0 0 20 0 2 0 338584291 372740096 7390 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9360] ppid=9346 vsize=364016 CPUtime=0.14 cores=1,3,5,7
/proc/9360/stat : 9360 (python) S 9346 9346 3958 0 -1 4202560 1156 0 0 0 7 7 0 0 20 0 2 0 338584294 372752384 7376 33554432000 4194304 5683644 140737024878048 140737024869696 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9360/statm: 91004 7376 383 364 0 26224 0
[pid=9362] ppid=9360 vsize=27760 CPUtime=92.21 cores=1,3,5,7
/proc/9362/stat : 9362 (clasp-1.3.7-x86) T 9360 9362 9362 0 -1 4202496 7171 0 0 0 9220 1 0 0 20 0 1 0 338584294 28426240 6683 33554432000 134512640 136139019 4287678240 4287677168 134649220 0 0 16781312 16386 18446744071579387992 0 0 17 3 0 0 0 0 0
/proc/9362/statm: 6940 6683 182 398 0 6540 0
[pid=9360/tid=9364] ppid=9346 vsize=364016 CPUtime=0 cores=1,3,5,7
/proc/9360/task/9364/stat : 9364 (python) S 9346 9346 3958 0 -1 4202560 52 0 0 0 0 0 0 0 20 0 2 0 338589007 372752384 7376 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9365] ppid=9346 vsize=439808 CPUtime=0.08 cores=1,3,5,7
/proc/9365/stat : 9365 (python) S 9346 9346 3958 0 -1 4202560 1119 0 0 0 4 4 0 0 20 0 2 0 338589009 450363392 7382 33554432000 4194304 5683644 140737024878048 140737024869696 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9365/statm: 109952 7382 383 364 0 45169 0
[pid=9367] ppid=9365 vsize=555828 CPUtime=50.53 cores=1,3,5,7
/proc/9367/stat : 9367 (scip-2.0.1.linu) T 9365 9367 9367 0 -1 4202496 134093 0 0 0 5009 44 0 0 20 0 1 0 338589009 569167872 133368 33554432000 4194304 13684479 140736677566768 140736677563080 4880915 0 0 16781312 2 18446744071579387992 0 0 17 5 0 0 0 0 0
/proc/9367/statm: 138957 133368 639 2317 0 136635 0
[pid=9365/tid=9390] ppid=9346 vsize=439808 CPUtime=0 cores=1,3,5,7
/proc/9365/task/9390/stat : 9390 (python) S 9346 9346 3958 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 338594064 450363392 7382 33554432000 4194304 5683644 140737024878048 139681815675440 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9392] ppid=9346 vsize=515600 CPUtime=0.06 cores=1,3,5,7
/proc/9392/stat : 9392 (python) S 9346 9346 3958 0 -1 4202560 1125 0 0 0 3 3 0 0 20 0 2 0 338598405 527974400 7895 33554432000 4194304 5683644 140737024878048 140737024869696 217413903629 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/9392/statm: 128900 7895 383 364 0 64114 0
[pid=9394] ppid=9392 vsize=9624972 CPUtime=46.29 cores=1,3,5,7
/proc/9394/stat : 9394 (java) T 9392 9394 9394 0 -1 4202496 19684 0 1 0 4596 33 0 0 20 0 19 0 338598405 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 140735871947760 217413877821 0 0 0 16800975 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9394/statm: 2406243 286658 2269 9 0 2396937 0
[pid=9394/tid=9395] ppid=9392 vsize=9624972 CPUtime=43.29 cores=1,3,5,7
/proc/9394/task/9395/stat : 9395 (java) T 9392 9394 9394 0 -1 4202560 2989 0 1 0 4313 16 0 0 20 0 19 0 338598407 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735826529360 139735699764747 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9394/tid=9396] ppid=9392 vsize=9624972 CPUtime=0.14 cores=1,3,5,7
/proc/9394/task/9396/stat : 9396 (java) T 9392 9394 9394 0 -1 4202560 122 0 0 0 12 2 0 0 20 0 19 0 338598408 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735670225704 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9394/tid=9397] ppid=9392 vsize=9624972 CPUtime=0.11 cores=1,3,5,7
/proc/9394/task/9397/stat : 9397 (java) T 9392 9394 9394 0 -1 4202560 33 0 0 0 9 2 0 0 20 0 19 0 338598408 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735669172904 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9394/tid=9398] ppid=9392 vsize=9624972 CPUtime=0.12 cores=1,3,5,7
/proc/9394/task/9398/stat : 9398 (java) T 9392 9394 9394 0 -1 4202560 40 0 0 0 10 2 0 0 20 0 19 0 338598408 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735668120616 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9394/tid=9399] ppid=9392 vsize=9624972 CPUtime=0.14 cores=1,3,5,7
/proc/9394/task/9399/stat : 9399 (java) T 9392 9394 9394 0 -1 4202560 113 0 0 0 11 3 0 0 20 0 19 0 338598408 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735667067816 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9394/tid=9400] ppid=9392 vsize=9624972 CPUtime=0.12 cores=1,3,5,7
/proc/9394/task/9400/stat : 9400 (java) T 9392 9394 9394 0 -1 4202560 58 0 0 0 10 2 0 0 20 0 19 0 338598408 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735666014504 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9394/tid=9401] ppid=9392 vsize=9624972 CPUtime=0.12 cores=1,3,5,7
/proc/9394/task/9401/stat : 9401 (java) T 9392 9394 9394 0 -1 4202560 44 0 0 0 10 2 0 0 20 0 19 0 338598408 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735664961704 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9394/tid=9402] ppid=9392 vsize=9624972 CPUtime=0.12 cores=1,3,5,7
/proc/9394/task/9402/stat : 9402 (java) T 9392 9394 9394 0 -1 4202560 99 0 0 0 10 2 0 0 20 0 19 0 338598408 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735663909416 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9394/tid=9403] ppid=9392 vsize=9624972 CPUtime=0.1 cores=1,3,5,7
/proc/9394/task/9403/stat : 9403 (java) T 9392 9394 9394 0 -1 4202560 54 0 0 0 10 0 0 0 20 0 19 0 338598408 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735662856616 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9394/tid=9404] ppid=9392 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9394/task/9404/stat : 9404 (java) T 9392 9394 9394 0 -1 4202560 943 0 0 0 0 0 0 0 20 0 19 0 338598409 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735650659048 217413892009 0 0 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9394/tid=9405] ppid=9392 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9394/task/9405/stat : 9405 (java) T 9392 9394 9394 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338598409 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735649605112 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9394/tid=9406] ppid=9392 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9394/task/9406/stat : 9406 (java) T 9392 9394 9394 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338598409 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735648552712 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9394/tid=9407] ppid=9392 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9394/task/9407/stat : 9407 (java) T 9392 9394 9394 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338598412 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735647501200 217413900064 0 4 0 16800975 18446744071579387992 0 0 -1 3 0 0 0 0 0
[pid=9394/tid=9408] ppid=9392 vsize=9624972 CPUtime=0.87 cores=1,3,5,7
/proc/9394/task/9408/stat : 9408 (java) T 9392 9394 9394 0 -1 4202560 4238 0 0 0 87 0 0 0 20 0 19 0 338598412 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735646447800 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=9394/tid=9409] ppid=9392 vsize=9624972 CPUtime=1 cores=1,3,5,7
/proc/9394/task/9409/stat : 9409 (java) T 9392 9394 9394 0 -1 4202560 9536 0 0 0 98 2 0 0 20 0 19 0 338598412 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735645395000 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9394/tid=9410] ppid=9392 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9394/task/9410/stat : 9410 (java) T 9392 9394 9394 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338598412 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735644342904 217413891132 0 4 0 16800975 18446744071579387992 0 0 -1 1 0 0 0 0 0
[pid=9394/tid=9411] ppid=9392 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9394/task/9411/stat : 9411 (java) T 9392 9394 9394 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338598412 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735643290312 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9394/tid=9412] ppid=9392 vsize=9624972 CPUtime=0 cores=1,3,5,7
/proc/9394/task/9412/stat : 9412 (java) T 9392 9394 9394 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338598463 9855971328 286658 33554432000 1073741824 1073778376 140735871956624 139735642236328 217413892009 0 4 0 16800975 18446744071579387992 0 0 -1 5 0 0 0 0 0
[pid=9392/tid=9482] ppid=9346 vsize=515600 CPUtime=0 cores=1,3,5,7
/proc/9392/task/9482/stat : 9482 (python) S 9346 9346 3958 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 2 0 338602772 527974400 7895 33554432000 4194304 5683644 140737024878048 139681731699248 217413900064 0 0 16781312 33554946 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9490] ppid=9346 vsize=591392 CPUtime=0 cores=1,3,5,7
/proc/9490/stat : 9490 (python) S 9346 9346 3958 0 -1 4202560 859 0 0 0 0 0 0 0 20 0 1 0 338607288 605585408 7853 33554432000 4194304 5683644 140737024878048 140737024869136 217410535907 0 0 16781312 33554946 18446744071580425449 0 0 17 3 0 0 0 0 0
/proc/9490/statm: 147848 7853 335 364 0 83059 0
[pid=9492] ppid=9490 vsize=9558408 CPUtime=0.35 cores=1,3,5,7
/proc/9492/stat : 9492 (java) S 9490 9492 9492 0 -1 4202496 6920 0 1 0 34 1 0 0 20 0 18 0 338607288 9787809792 8665 33554432000 1073741824 1073778376 140734292287552 140734292278688 217413877821 0 0 0 16800975 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/9492/statm: 2389602 8665 2205 9 0 2380296 0
[pid=9492/tid=9493] ppid=9490 vsize=9558408 CPUtime=0.19 cores=1,3,5,7
/proc/9492/task/9493/stat : 9493 (java) R 9490 9492 9492 0 -1 4202560 2420 0 1 0 19 0 0 0 20 0 18 0 338607290 9787809792 8666 33554432000 1073741824 1073778376 140734292287552 140266555592760 140266562744166 0 4 0 16800975 0 0 0 -1 5 0 0 0 0 0
[pid=9492/tid=9494] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9494/stat : 9494 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607291 9787809792 8666 33554432000 1073741824 1073778376 140734292287552 140266528045608 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9492/tid=9495] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9495/stat : 9495 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607291 9787809792 8666 33554432000 1073741824 1073778376 140734292287552 140266526992808 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=9492/tid=9496] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9496/stat : 9496 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607291 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140266525940008 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9492/tid=9497] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9497/stat : 9497 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607291 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140266524887208 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9492/tid=9498] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9498/stat : 9498 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607291 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140266523835432 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9492/tid=9499] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9499/stat : 9499 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607291 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140266522782632 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9492/tid=9500] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9500/stat : 9500 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607291 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140266521729832 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9492/tid=9501] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9501/stat : 9501 (java) S 9490 9492 9492 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 18 0 338607291 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140266520677032 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9492/tid=9502] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9502/stat : 9502 (java) S 9490 9492 9492 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 18 0 338607292 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265909098984 217413892009 0 0 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9492/tid=9503] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9503/stat : 9503 (java) S 9490 9492 9492 0 -1 4202560 16 0 0 0 0 0 0 0 20 0 18 0 338607292 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265908045048 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9492/tid=9504] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9504/stat : 9504 (java) S 9490 9492 9492 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 18 0 338607292 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265906992136 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9492/tid=9505] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9505/stat : 9505 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607295 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265905940624 217413900064 0 4 0 16800975 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9492/tid=9506] ppid=9490 vsize=9558408 CPUtime=0.07 cores=1,3,5,7
/proc/9492/task/9506/stat : 9506 (java) R 9490 9492 9492 0 -1 4202560 873 0 0 0 7 0 0 0 20 0 18 0 338607295 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265904876040 140266557683696 0 4 0 16800975 0 0 0 -1 3 0 0 0 0 0
[pid=9492/tid=9507] ppid=9490 vsize=9558408 CPUtime=0.06 cores=1,3,5,7
/proc/9492/task/9507/stat : 9507 (java) R 9490 9492 9492 0 -1 4202560 2216 0 0 0 6 0 0 0 20 0 18 0 338607295 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265903823240 140266559681448 0 4 0 16800975 0 0 0 -1 1 0 0 0 0 0
[pid=9492/tid=9508] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9508/stat : 9508 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607295 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265902783352 217413891132 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=9492/tid=9509] ppid=9490 vsize=9558408 CPUtime=0 cores=1,3,5,7
/proc/9492/task/9509/stat : 9509 (java) S 9490 9492 9492 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 18 0 338607295 9787809792 8667 33554432000 1073741824 1073778376 140734292287552 140265901730760 217413892009 0 4 0 16800975 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=9346/tid=9353] ppid=9344 vsize=667172 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9353/stat : 9353 (python) S 9344 9346 3958 0 -1 4202560 30 1010 0 0 0 0 0 0 20 0 6 0 338579329 683184128 9435 33554432000 4194304 5683644 140737024878048 139681826165296 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=9346/tid=9361] ppid=9344 vsize=667172 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9361/stat : 9361 (python) S 9344 9346 3958 0 -1 4202560 27 1010 0 0 0 0 0 0 20 0 6 0 338584294 683184128 9435 33554432000 4194304 5683644 140737024878048 139681815675440 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9346/tid=9366] ppid=9344 vsize=667172 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9366/stat : 9366 (python) S 9344 9346 3958 0 -1 4202560 19 1010 0 0 0 0 0 0 20 0 6 0 338589009 683184128 9435 33554432000 4194304 5683644 140737024878048 139681731699248 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=9346/tid=9393] ppid=9344 vsize=667172 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9393/stat : 9393 (python) S 9344 9346 3958 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 6 0 338598405 683184128 9435 33554432000 4194304 5683644 140737024878048 139681721209392 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=9346/tid=9491] ppid=9344 vsize=667172 CPUtime=0 cores=1,3,5,7
/proc/9346/task/9491/stat : 9491 (python) S 9344 9346 3958 0 -1 4202560 21 1010 0 0 0 0 0 0 20 0 6 0 338607288 683184128 9435 33554432000 4194304 5683644 140737024878048 139681710719536 217413900064 0 0 16781312 33554434 18446744071579510697 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 283.81
Current children cumulated vsize (KiB) 23483600

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 9346 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=955854
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=115982
# CPU time returned by wait4() is 1.07184
# while last known CPU time is 283.81
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 281.555
CPU time (s): 283.81
CPU user time (s): 281.99
CPU system time (s): 1.82
CPU usage (%): 100.801
Max. virtual memory (cumulated for all children) (KiB): 23483600

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.955854
system time used= 0.115982
maximum resident set size= 37740
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 17867
page faults= 1
swaps= 0
block input operations= 0
block output operations= 448
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2082
involuntary context switches= 13

runsolver used 0.902862 second user time and 2.49462 second system time

The end

Launcher Data 

Begin job on node122 at 2012-06-03 13:55:30
IDJOB=3742343
IDBENCH=90161
IDSOLVER=2327
FILE ID=node122/3742343-1338724530
RUNJOBID= node122-1338713392-3975
PBS_JOBID= 14636732
Free space on /tmp= 71084 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K129.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742343-1338724530/watcher-3742343-1338724530 -o /tmp/evaluation-result-3742343-1338724530/solver-3742343-1338724530 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742343-1338724530.opb -seed 639862488 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1a22ba01765c232fe0c8623cfb5d750f
RANDOM SEED=639862488

node122.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5438.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.533
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        30861008 kB
Buffers:          208324 kB
Cached:           376148 kB
SwapCached:         1804 kB
Active:          1135700 kB
Inactive:          40232 kB
Active(anon):     586644 kB
Inactive(anon):     7052 kB
Active(file):     549056 kB
Inactive(file):    33180 kB
Unevictable:        7628 kB
Mlocked:            7628 kB
SwapTotal:      67108856 kB
SwapFree:       67100164 kB
Dirty:            102984 kB
Writeback:             0 kB
AnonPages:        597392 kB
Mapped:            18420 kB
Shmem:                36 kB
Slab:             692036 kB
SReclaimable:     131436 kB
SUnreclaim:       560600 kB
KernelStack:        1504 kB
PageTables:         6800 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:     828180 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70984 MiB
End job on node122 at 2012-06-03 14:00:15