Trace number 3742356

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB11: borg pb-dec-11.04.03? (MO) 287.29 284.455

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K192.opb
MD5SUM83a21ae30c37a390d06f7aafb55d6d81
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables22848
Total number of constraints23234
Number of constraints which are clauses23042
Number of constraints which are cardinality constraints (but not clauses)192
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint192
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 193
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.43/1.51	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.43/1.51	c INFO: solving HOME/instance-3742356-1338724544.opb
0.43/1.78	c INFO: parsing took 0.26 s
0.43/2.28	c INFO: features took 0.50 CPU seconds
0.43/2.31	c INFO: running scip-2.0.1-spx@0 for 45 with 1798 remaining (b = 0.58)
51.06/52.14	c INFO: running clasp-1.3.7@0 for 45 with 1749 remaining (b = 0.46)
98.31/99.30	c INFO: running scip-2.0.1-clp@0 for 45 with 1702 remaining (b = 0.08)
147.38/148.30	c INFO: running scip-2.0.1-clp@48 for 45 with 1653 remaining (b = 0.01)
193.56/194.49	c INFO: running sat4j-pb-v20101225-cutting@0 for 45 with 1607 remaining (b = 0.04)
240.02/238.26	c INFO: running clasp-1.3.7@47 for 45 with 1562 remaining (b = 0.01)
285.33/283.42	c INFO: running sat4j-pb-v20101225@0 for 45 with 1516 remaining (b = 0.17)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742356-1338724544/watcher-3742356-1338724544 -o /tmp/evaluation-result-3742356-1338724544/solver-3742356-1338724544 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742356-1338724544.opb -seed 1281393731 -budget 1800 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.93 2.01 2.09 4/187 7733
/proc/meminfo: memFree=30229012/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=15276 CPUtime=0 cores=0,2,4,6
/proc/7733/stat : 7733 (python) R 7731 7733 2614 0 -1 4202496 633 0 0 0 0 0 0 0 20 0 1 0 338579461 15642624 526 33554432000 4194304 5683644 140734328141872 140734328123544 240565175184 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7733/statm: 3819 526 311 364 0 261 0

[startup+0.0108809 s]
/proc/loadavg: 1.93 2.01 2.09 4/187 7733
/proc/meminfo: memFree=30229012/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=15380 CPUtime=0 cores=0,2,4,6
/proc/7733/stat : 7733 (python) D 7731 7733 2614 0 -1 4202496 740 0 0 0 0 0 0 0 20 0 1 0 338579461 15749120 617 33554432000 4194304 5683644 140734328141872 140734328100328 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/7733/statm: 3845 617 331 364 0 287 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15380

[startup+0.100232 s]
/proc/loadavg: 1.93 2.01 2.09 4/187 7733
/proc/meminfo: memFree=30229012/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=19084 CPUtime=0.03 cores=0,2,4,6
/proc/7733/stat : 7733 (python) R 7731 7733 2614 0 -1 4202496 1397 0 0 0 3 0 0 0 20 0 1 0 338579461 19542016 1151 33554432000 4194304 5683644 140734328141872 140734328076808 4509181 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7733/statm: 4771 1151 430 364 0 695 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 19084

[startup+0.300236 s]
/proc/loadavg: 1.93 2.01 2.09 4/187 7733
/proc/meminfo: memFree=30229012/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=96892 CPUtime=0.12 cores=0,2,4,6
/proc/7733/stat : 7733 (python) D 7731 7733 2614 0 -1 4202496 3910 1011 1 0 10 2 0 0 20 0 1 0 338579461 99217408 2647 33554432000 4194304 5683644 140734328141872 140734328041864 240565175184 0 0 16781312 2 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/7733/statm: 24223 2647 822 364 0 1714 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 96892

[startup+0.700348 s]
/proc/loadavg: 1.93 2.01 2.09 4/187 7733
/proc/meminfo: memFree=30229012/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=122344 CPUtime=0.23 cores=0,2,4,6
/proc/7733/stat : 7733 (python) R 7731 7733 2614 0 -1 4202496 5973 1011 1 0 17 6 0 0 20 0 1 0 338579461 125280256 3849 33554432000 4194304 5683644 140734328141872 140734328057272 240565175184 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7733/statm: 30586 3849 1015 364 0 2829 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 122344

[startup+1.50023 s]
/proc/loadavg: 1.93 2.01 2.09 3/188 7738
/proc/meminfo: memFree=30206204/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=272044 CPUtime=0.43 cores=0,2,4,6
/proc/7733/stat : 7733 (python) R 7731 7733 2614 0 -1 4202496 9659 1011 1 0 31 12 0 0 20 0 1 0 338579461 278573056 6780 33554432000 4194304 5683644 140734328141872 140734328055944 4539725 0 0 16781312 2 0 0 0 17 0 0 0 21 0 0
/proc/7733/statm: 68011 6780 1749 364 0 5380 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 272044

[startup+3.10033 s]
/proc/loadavg: 2.02 2.03 2.10 3/191 7741
/proc/meminfo: memFree=30094084/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=367628 CPUtime=1.24 cores=0,2,4,6
/proc/7733/stat : 7733 (python) S 7731 7733 2614 0 -1 4202496 14523 1011 1 0 111 13 0 0 20 0 2 0 338579461 376451072 9770 33554432000 4194304 5683644 140734328141872 140734328136704 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 21 0 0
/proc/7733/statm: 91907 9770 1902 364 0 27130 0
[pid=7739] ppid=7733 vsize=291848 CPUtime=0 cores=0,2,4,6
/proc/7739/stat : 7739 (python) S 7733 7733 2614 0 -1 4202560 744 0 0 0 0 0 0 0 20 0 1 0 338579692 298852352 8202 33554432000 4194304 5683644 140734328141872 140734328132960 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 4 0 0 0 0 0
/proc/7739/statm: 72962 8202 340 364 0 8185 0
[pid=7741] ppid=7739 vsize=132500 CPUtime=0.77 cores=0,2,4,6
/proc/7741/stat : 7741 (scip-2.0.1.linu) R 7739 7741 7741 0 -1 4202496 30765 0 0 0 71 6 0 0 20 0 1 0 338579692 135680000 30048 33554432000 4194304 13684479 140734757217856 140734757213240 4880958 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7741/statm: 33125 30048 622 2317 0 30803 0
[pid=7733/tid=7740] ppid=7731 vsize=367628 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7740/stat : 7740 (python) S 7731 7733 2614 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 2 0 338579692 376451072 9770 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 2.01
Current children cumulated vsize (KiB) 791976
heavy processes:

[startup+6.3005 s]
/proc/loadavg: 2.02 2.03 2.10 3/191 7741
/proc/meminfo: memFree=29972440/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=367628 CPUtime=1.24 cores=0,2,4,6
/proc/7733/stat : 7733 (python) S 7731 7733 2614 0 -1 4202496 14523 1011 1 0 111 13 0 0 20 0 2 0 338579461 376451072 9770 33554432000 4194304 5683644 140734328141872 140734328136704 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 21 0 0
/proc/7733/statm: 91907 9770 1902 364 0 27130 0
[pid=7739] ppid=7733 vsize=291848 CPUtime=0 cores=0,2,4,6
/proc/7739/stat : 7739 (python) S 7733 7733 2614 0 -1 4202560 892 0 0 0 0 0 0 0 20 0 1 0 338579692 298852352 8225 33554432000 4194304 5683644 140734328141872 140734328132960 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/7739/statm: 72962 8225 363 364 0 8185 0
[pid=7741] ppid=7739 vsize=212964 CPUtime=3.97 cores=0,2,4,6
/proc/7741/stat : 7741 (scip-2.0.1.linu) R 7739 7741 7741 0 -1 4202496 49531 0 0 0 388 9 0 0 20 0 1 0 338579692 218075136 48814 33554432000 4194304 13684479 140734757217856 140734757213240 4880933 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7741/statm: 53241 48814 622 2317 0 50919 0
[pid=7733/tid=7740] ppid=7731 vsize=367628 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7740/stat : 7740 (python) S 7731 7733 2614 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 2 0 338579692 376451072 9770 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 5.21
Current children cumulated vsize (KiB) 872440

[startup+12.7004 s]
/proc/loadavg: 2.01 2.03 2.09 3/191 7741
/proc/meminfo: memFree=29872800/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=367628 CPUtime=1.24 cores=0,2,4,6
/proc/7733/stat : 7733 (python) S 7731 7733 2614 0 -1 4202496 14523 1011 1 0 111 13 0 0 20 0 2 0 338579461 376451072 9770 33554432000 4194304 5683644 140734328141872 140734328136704 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 21 0 0
/proc/7733/statm: 91907 9770 1902 364 0 27130 0
[pid=7739] ppid=7733 vsize=291848 CPUtime=0 cores=0,2,4,6
/proc/7739/stat : 7739 (python) S 7733 7733 2614 0 -1 4202560 897 0 0 0 0 0 0 0 20 0 1 0 338579692 298852352 8225 33554432000 4194304 5683644 140734328141872 140734328132960 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/7739/statm: 72962 8225 363 364 0 8185 0
[pid=7741] ppid=7739 vsize=285704 CPUtime=10.38 cores=0,2,4,6
/proc/7741/stat : 7741 (scip-2.0.1.linu) R 7739 7741 7741 0 -1 4202496 66613 0 0 0 1026 12 0 0 20 0 1 0 338579692 292560896 65896 33554432000 4194304 13684479 140734757217856 140734757213240 4880947 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7741/statm: 71426 65896 622 2317 0 69104 0
[pid=7733/tid=7740] ppid=7731 vsize=367628 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7740/stat : 7740 (python) S 7731 7733 2614 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 2 0 338579692 376451072 9770 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 11.62
Current children cumulated vsize (KiB) 945180

[startup+25.5004 s]
/proc/loadavg: 2.01 2.03 2.09 3/192 7742
/proc/meminfo: memFree=29715596/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=367628 CPUtime=1.24 cores=0,2,4,6
/proc/7733/stat : 7733 (python) S 7731 7733 2614 0 -1 4202496 14523 1011 1 0 111 13 0 0 20 0 2 0 338579461 376451072 9770 33554432000 4194304 5683644 140734328141872 140734328136704 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 21 0 0
/proc/7733/statm: 91907 9770 1902 364 0 27130 0
[pid=7739] ppid=7733 vsize=291848 CPUtime=0.03 cores=0,2,4,6
/proc/7739/stat : 7739 (python) S 7733 7733 2614 0 -1 4202560 907 0 0 0 2 1 0 0 20 0 1 0 338579692 298852352 8225 33554432000 4194304 5683644 140734328141872 140734328132960 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 2 0 0 0 0 0
/proc/7739/statm: 72962 8225 363 364 0 8185 0
[pid=7741] ppid=7739 vsize=391748 CPUtime=23.17 cores=0,2,4,6
/proc/7741/stat : 7741 (scip-2.0.1.linu) R 7739 7741 7741 0 -1 4202496 92399 0 0 0 2298 19 0 0 20 0 1 0 338579692 401149952 91950 33554432000 4194304 13684479 140734757217856 140734757213240 4880943 0 0 16781312 2 0 0 0 17 0 0 0 0 0 0
/proc/7741/statm: 97937 91950 635 2317 0 95615 0
[pid=7733/tid=7740] ppid=7731 vsize=367628 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7740/stat : 7740 (python) S 7731 7733 2614 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 2 0 338579692 376451072 9770 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 24.44
Current children cumulated vsize (KiB) 1051224

[startup+51.1004 s]
/proc/loadavg: 2.07 2.04 2.09 3/192 7742
/proc/meminfo: memFree=29424192/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=367628 CPUtime=1.24 cores=0,2,4,6
/proc/7733/stat : 7733 (python) S 7731 7733 2614 0 -1 4202496 14523 1011 1 0 111 13 0 0 20 0 2 0 338579461 376451072 9770 33554432000 4194304 5683644 140734328141872 140734328136704 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 0 0 0 21 0 0
/proc/7733/statm: 91907 9770 1902 364 0 27130 0
[pid=7739] ppid=7733 vsize=291848 CPUtime=0.06 cores=0,2,4,6
/proc/7739/stat : 7739 (python) S 7733 7733 2614 0 -1 4202560 1016 0 0 0 4 2 0 0 20 0 1 0 338579692 298852352 8230 33554432000 4194304 5683644 140734328141872 140734328132960 240565191080 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/7739/statm: 72962 8230 368 364 0 8185 0
[pid=7741] ppid=7739 vsize=580580 CPUtime=48.76 cores=0,2,4,6
/proc/7741/stat : 7741 (scip-2.0.1.linu) R 7739 7741 7741 0 -1 4202496 139132 0 0 0 4842 34 0 0 20 0 1 0 338579692 594513920 139194 33554432000 4194304 13684479 140734757217856 140734757214168 11876362 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/7741/statm: 145145 139194 639 2317 0 142823 0
[pid=7733/tid=7740] ppid=7731 vsize=367628 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7740/stat : 7740 (python) S 7731 7733 2614 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 2 0 338579692 376451072 9770 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 50.06
Current children cumulated vsize (KiB) 1240056

[startup+102.301 s]
/proc/loadavg: 2.03 2.03 2.09 3/221 7771
/proc/meminfo: memFree=27448896/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=519212 CPUtime=1.29 cores=0,2,4,6
/proc/7733/stat : 7733 (python) S 7731 7733 2614 0 -1 4202496 16394 1011 1 0 116 13 0 0 20 0 4 0 338579461 531673088 9787 33554432000 4194304 5683644 140734328141872 140734328136704 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 6 0 0 21 0 0
/proc/7733/statm: 129803 9787 1907 364 0 65020 0
[pid=7739] ppid=7733 vsize=367628 CPUtime=0.07 cores=0,2,4,6
/proc/7739/stat : 7739 (python) S 7733 7733 2614 0 -1 4202560 1157 0 0 0 5 2 0 0 20 0 2 0 338579692 376451072 8265 33554432000 4194304 5683644 140734328141872 140734328133520 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7739/statm: 91907 8265 396 364 0 27130 0
[pid=7741] ppid=7739 vsize=592836 CPUtime=49.78 cores=0,2,4,6
/proc/7741/stat : 7741 (scip-2.0.1.linu) T 7739 7741 7741 0 -1 4202496 142249 0 0 0 4943 35 0 0 20 0 1 0 338579692 607064064 142311 33554432000 4194304 13684479 140734757217856 140734757214152 5563256 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7741/statm: 148209 142311 639 2317 0 145887 0
[pid=7739/tid=7743] ppid=7733 vsize=367628 CPUtime=0 cores=0,2,4,6
/proc/7739/task/7743/stat : 7743 (python) S 7733 7733 2614 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 338584673 376451072 8265 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7744] ppid=7733 vsize=367640 CPUtime=0.07 cores=0,2,4,6
/proc/7744/stat : 7744 (python) S 7733 7733 2614 0 -1 4202560 1096 0 0 0 4 3 0 0 20 0 2 0 338584675 376463360 8252 33554432000 4194304 5683644 140734328141872 140734328133520 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7744/statm: 91910 8252 383 364 0 27130 0
[pid=7746] ppid=7744 vsize=12624 CPUtime=47.11 cores=0,2,4,6
/proc/7746/stat : 7746 (clasp-1.3.7-x86) T 7744 7746 7746 0 -1 4202496 3494 0 0 0 4711 0 0 0 20 0 1 0 338584676 12926976 2886 33554432000 134512640 136139019 4291722544 4291721520 134883407 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7746/statm: 3156 2886 182 398 0 2756 0
[pid=7744/tid=7768] ppid=7733 vsize=367640 CPUtime=0 cores=0,2,4,6
/proc/7744/task/7768/stat : 7768 (python) S 7733 7733 2614 0 -1 4202560 54 0 0 0 0 0 0 0 20 0 2 0 338589389 376463360 8252 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7769] ppid=7733 vsize=443432 CPUtime=0 cores=0,2,4,6
/proc/7769/stat : 7769 (python) S 7733 7733 2614 0 -1 4202560 797 0 0 0 0 0 0 0 20 0 1 0 338589391 454074368 8209 33554432000 4194304 5683644 140734328141872 140734328132960 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 0 0 0 0 0 0
/proc/7769/statm: 110858 8209 335 364 0 46075 0
[pid=7771] ppid=7769 vsize=212152 CPUtime=2.98 cores=0,2,4,6
/proc/7771/stat : 7771 (scip-2.0.1.linu) R 7769 7771 7771 0 -1 4202496 49004 0 0 0 291 7 0 0 20 0 1 0 338589392 217243648 48288 33554432000 4194304 15243407 140733891536768 140733891532152 4880915 0 0 16781312 2 0 0 0 17 4 0 0 0 0 0
/proc/7771/statm: 53038 48288 669 2698 0 50333 0
[pid=7733/tid=7740] ppid=7731 vsize=519212 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7740/stat : 7740 (python) S 7731 7733 2614 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 4 0 338579692 531673088 9787 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7733/tid=7745] ppid=7731 vsize=519212 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7745/stat : 7745 (python) S 7731 7733 2614 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 4 0 338584675 531673088 9787 33554432000 4194304 5683644 140734328141872 140272208495152 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7733/tid=7770] ppid=7731 vsize=519212 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7770/stat : 7770 (python) S 7731 7733 2614 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 4 0 338589391 531673088 9787 33554432000 4194304 5683644 140734328141872 140272198005296 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 101.3
Current children cumulated vsize (KiB) 2515524

[startup+162.301 s]
/proc/loadavg: 2.01 2.02 2.08 3/192 7842
/proc/meminfo: memFree=29515640/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=519212 CPUtime=1.31 cores=0,2,4,6

################
# More data... #
################

[pid=7878/tid=7892] ppid=7876 vsize=9624972 CPUtime=0.47 cores=0,2,4,6
/proc/7878/task/7892/stat : 7892 (java) S 7876 7878 7878 0 -1 4202560 2564 0 0 0 47 0 0 0 20 0 19 0 338607811 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803077094072 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=7878/tid=7893] ppid=7876 vsize=9624972 CPUtime=0.46 cores=0,2,4,6
/proc/7878/task/7893/stat : 7893 (java) S 7876 7878 7878 0 -1 4202560 3079 0 0 0 46 0 0 0 20 0 19 0 338607811 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803076041272 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7894] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7894/stat : 7894 (java) S 7876 7878 7878 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338607811 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803074989176 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7895] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7895/stat : 7895 (java) S 7876 7878 7878 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338607811 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803073936584 240568547241 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7896] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7896/stat : 7896 (java) S 7876 7878 7878 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338607871 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803072881576 240568547241 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7733/tid=7740] ppid=7731 vsize=670796 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7740/stat : 7740 (python) S 7731 7733 2614 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 6 0 338579692 686895104 9803 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7733/tid=7745] ppid=7731 vsize=670796 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7745/stat : 7745 (python) S 7731 7733 2614 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 6 0 338584675 686895104 9803 33554432000 4194304 5683644 140734328141872 140272208495152 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7733/tid=7770] ppid=7731 vsize=670796 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7770/stat : 7770 (python) S 7731 7733 2614 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 6 0 338589391 686895104 9803 33554432000 4194304 5683644 140734328141872 140272198005296 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7733/tid=7844] ppid=7731 vsize=670796 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7844/stat : 7844 (python) S 7731 7733 2614 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 6 0 338598910 686895104 9803 33554432000 4194304 5683644 140734328141872 140272187515440 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7733/tid=7877] ppid=7731 vsize=670796 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7877/stat : 7877 (python) S 7731 7733 2614 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 6 0 338607803 686895104 9803 33554432000 4194304 5683644 140734328141872 140272177025584 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 287.29
Current children cumulated vsize (KiB) 23660004

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+284.401 s]
/proc/loadavg: 2.03 2.03 2.08 3/244 7896
/proc/meminfo: memFree=26319596/32873844 swapFree=7116/7116
[pid=7733] ppid=7731 vsize=670796 CPUtime=1.39 cores=0,2,4,6
/proc/7733/stat : 7733 (python) S 7731 7733 2614 0 -1 4202496 18371 1011 1 0 125 14 0 0 20 0 6 0 338579461 686895104 9803 33554432000 4194304 5683644 140734328141872 140734328136704 240568558861 0 0 16781312 33554434 18446744073709551615 0 0 17 4 0 0 21 0 0
/proc/7733/statm: 167699 9803 1911 364 0 102910 0
[pid=7739] ppid=7733 vsize=367628 CPUtime=0.07 cores=0,2,4,6
/proc/7739/stat : 7739 (python) S 7733 7733 2614 0 -1 4202560 1157 0 0 0 5 2 0 0 20 0 2 0 338579692 376451072 8265 33554432000 4194304 5683644 140734328141872 140734328133520 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7739/statm: 91907 8265 396 364 0 27130 0
[pid=7741] ppid=7739 vsize=592836 CPUtime=49.78 cores=0,2,4,6
/proc/7741/stat : 7741 (scip-2.0.1.linu) T 7739 7741 7741 0 -1 4202496 142249 0 0 0 4943 35 0 0 20 0 1 0 338579692 607064064 143383 33554432000 4194304 13684479 140734757217856 140734757214152 5563256 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7741/statm: 148209 143383 639 2317 0 145887 0
[pid=7739/tid=7743] ppid=7733 vsize=367628 CPUtime=0 cores=0,2,4,6
/proc/7739/task/7743/stat : 7743 (python) S 7733 7733 2614 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 2 0 338584673 376451072 8265 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7744] ppid=7733 vsize=367640 CPUtime=0.17 cores=0,2,4,6
/proc/7744/stat : 7744 (python) S 7733 7733 2614 0 -1 4202560 1156 0 0 0 10 7 0 0 20 0 2 0 338584675 376463360 8252 33554432000 4194304 5683644 140734328141872 140734328133520 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7744/statm: 91910 8252 383 364 0 27130 0
[pid=7746] ppid=7744 vsize=14844 CPUtime=92.24 cores=0,2,4,6
/proc/7746/stat : 7746 (clasp-1.3.7-x86) T 7744 7746 7746 0 -1 4202496 4076 0 0 0 9224 0 0 0 20 0 1 0 338584676 15200256 3468 33554432000 134512640 136139019 4291722544 4291721472 134891505 0 0 16781312 16386 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7746/statm: 3711 3468 182 398 0 3311 0
[pid=7744/tid=7768] ppid=7733 vsize=367640 CPUtime=0 cores=0,2,4,6
/proc/7744/task/7768/stat : 7768 (python) S 7733 7733 2614 0 -1 4202560 54 0 0 0 0 0 0 0 20 0 2 0 338589389 376463360 8252 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7769] ppid=7733 vsize=443432 CPUtime=0.17 cores=0,2,4,6
/proc/7769/stat : 7769 (python) S 7733 7733 2614 0 -1 4202560 1164 0 0 0 9 8 0 0 20 0 2 0 338589391 454074368 8258 33554432000 4194304 5683644 140734328141872 140734328133520 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7769/statm: 110858 8258 383 364 0 46075 0
[pid=7771] ppid=7769 vsize=838644 CPUtime=95.11 cores=0,2,4,6
/proc/7771/stat : 7771 (scip-2.0.1.linu) T 7769 7771 7771 0 -1 4202496 203540 0 0 0 9439 72 0 0 20 0 1 0 338589392 858771456 202538 33554432000 4194304 15243407 140733891536768 140733891532856 13277498 0 0 16781312 2 18446744071579387992 0 0 17 4 0 0 0 0 0
/proc/7771/statm: 209661 202538 885 2698 0 206956 0
[pid=7769/tid=7774] ppid=7733 vsize=443432 CPUtime=0 cores=0,2,4,6
/proc/7769/task/7774/stat : 7774 (python) S 7733 7733 2614 0 -1 4202560 51 0 0 0 0 0 0 0 20 0 2 0 338594288 454074368 8258 33554432000 4194304 5683644 140734328141872 140272208495152 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7843] ppid=7733 vsize=519224 CPUtime=0.07 cores=0,2,4,6
/proc/7843/stat : 7843 (python) S 7733 7733 2614 0 -1 4202560 1150 0 0 0 4 3 0 0 20 0 2 0 338598910 531685376 8264 33554432000 4194304 5683644 140734328141872 140734328133520 240568558861 0 0 16781312 33554946 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7843/statm: 129806 8264 383 364 0 65020 0
[pid=7845] ppid=7843 vsize=9624972 CPUtime=46.34 cores=0,2,4,6
/proc/7845/stat : 7845 (java) T 7843 7845 7845 0 -1 4202496 20409 0 1 0 4577 57 0 0 20 0 19 0 338598910 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 140734353915968 240568533053 0 0 0 16800975 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/7845/statm: 2406243 542711 2268 9 0 2396937 0
[pid=7845/tid=7846] ppid=7843 vsize=9624972 CPUtime=43.28 cores=0,2,4,6
/proc/7845/task/7846/stat : 7846 (java) T 7843 7845 7845 0 -1 4202560 3921 0 1 0 4294 34 0 0 20 0 19 0 338598911 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917467675616 139917301618348 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7847] ppid=7843 vsize=9624972 CPUtime=0.12 cores=0,2,4,6
/proc/7845/task/7847/stat : 7847 (java) T 7843 7845 7845 0 -1 4202560 64 0 0 0 9 3 0 0 20 0 19 0 338598912 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917429168552 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7848] ppid=7843 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/7845/task/7848/stat : 7848 (java) T 7843 7845 7845 0 -1 4202560 35 0 0 0 6 2 0 0 20 0 19 0 338598912 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917428116008 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7849] ppid=7843 vsize=9624972 CPUtime=0.14 cores=0,2,4,6
/proc/7845/task/7849/stat : 7849 (java) T 7843 7845 7845 0 -1 4202560 71 0 0 0 11 3 0 0 20 0 19 0 338598912 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917427062952 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7850] ppid=7843 vsize=9624972 CPUtime=0.13 cores=0,2,4,6
/proc/7845/task/7850/stat : 7850 (java) T 7843 7845 7845 0 -1 4202560 67 0 0 0 10 3 0 0 20 0 19 0 338598912 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917426010408 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7851] ppid=7843 vsize=9624972 CPUtime=0.08 cores=0,2,4,6
/proc/7845/task/7851/stat : 7851 (java) T 7843 7845 7845 0 -1 4202560 44 0 0 0 6 2 0 0 20 0 19 0 338598912 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917424958376 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7852] ppid=7843 vsize=9624972 CPUtime=0.14 cores=0,2,4,6
/proc/7845/task/7852/stat : 7852 (java) T 7843 7845 7845 0 -1 4202560 62 0 0 0 11 3 0 0 20 0 19 0 338598912 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917423905832 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7853] ppid=7843 vsize=9624972 CPUtime=0.09 cores=0,2,4,6
/proc/7845/task/7853/stat : 7853 (java) T 7843 7845 7845 0 -1 4202560 48 0 0 0 7 2 0 0 20 0 19 0 338598912 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917422852776 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7854] ppid=7843 vsize=9624972 CPUtime=0.1 cores=0,2,4,6
/proc/7845/task/7854/stat : 7854 (java) T 7843 7845 7845 0 -1 4202560 59 0 0 0 8 2 0 0 20 0 19 0 338598912 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917421800232 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7845/tid=7855] ppid=7843 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7845/task/7855/stat : 7855 (java) T 7843 7845 7845 0 -1 4202560 1267 0 0 0 0 0 0 0 20 0 19 0 338598913 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917291116904 240568547241 0 0 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7845/tid=7856] ppid=7843 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7845/task/7856/stat : 7856 (java) T 7843 7845 7845 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338598913 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917290063224 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7857] ppid=7843 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7845/task/7857/stat : 7857 (java) T 7843 7845 7845 0 -1 4202560 9 0 0 0 0 0 0 0 20 0 19 0 338598913 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917289010056 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7858] ppid=7843 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7845/task/7858/stat : 7858 (java) T 7843 7845 7845 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338598916 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917287958800 240568555296 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7859] ppid=7843 vsize=9624972 CPUtime=1 cores=0,2,4,6
/proc/7845/task/7859/stat : 7859 (java) T 7843 7845 7845 0 -1 4202560 4541 0 0 0 100 0 0 0 20 0 19 0 338598916 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917286906680 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7860] ppid=7843 vsize=9624972 CPUtime=0.9 cores=0,2,4,6
/proc/7845/task/7860/stat : 7860 (java) T 7843 7845 7845 0 -1 4202560 8810 0 0 0 88 2 0 0 20 0 19 0 338598916 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139917285854136 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7861] ppid=7843 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7845/task/7861/stat : 7861 (java) T 7843 7845 7845 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338598916 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139916411398904 240568546364 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7845/tid=7862] ppid=7843 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7845/task/7862/stat : 7862 (java) T 7843 7845 7845 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338598916 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139916410346568 240568547241 0 4 0 16800975 18446744071579387992 0 0 -1 0 0 0 0 0 0
[pid=7845/tid=7863] ppid=7843 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7845/task/7863/stat : 7863 (java) T 7843 7845 7845 0 -1 4202560 12 0 0 0 0 0 0 0 20 0 19 0 338598975 9855971328 542711 33554432000 1073741824 1073778376 140734353924832 139916409291816 240568547241 0 4 0 16800975 18446744071579387992 0 0 -1 2 0 0 0 0 0
[pid=7843/tid=7868] ppid=7733 vsize=519224 CPUtime=0 cores=0,2,4,6
/proc/7843/task/7868/stat : 7868 (python) S 7733 7733 2614 0 -1 4202560 51 0 0 0 0 0 0 0 20 0 2 0 338603285 531685376 8264 33554432000 4194304 5683644 140734328141872 140272198005296 240568555296 0 0 16781312 33554946 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7876] ppid=7733 vsize=595016 CPUtime=0 cores=0,2,4,6
/proc/7876/stat : 7876 (python) S 7733 7733 2614 0 -1 4202560 859 0 0 0 0 0 0 0 20 0 1 0 338607803 609296384 8221 33554432000 4194304 5683644 140734328141872 140734328132960 240565191139 0 0 16781312 33554946 18446744071580425449 0 0 17 6 0 0 0 0 0
/proc/7876/statm: 148754 8221 335 364 0 83965 0
[pid=7878] ppid=7876 vsize=9624972 CPUtime=1.95 cores=0,2,4,6
/proc/7878/stat : 7878 (java) S 7876 7878 7878 0 -1 4202496 9661 0 1 0 189 6 0 0 20 0 19 0 338607803 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 140736076813616 240568533053 0 0 0 16800975 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/7878/statm: 2406243 42481 2264 9 0 2396937 0
[pid=7878/tid=7879] ppid=7876 vsize=9624972 CPUtime=0.93 cores=0,2,4,6
/proc/7878/task/7879/stat : 7879 (java) R 7876 7878 7878 0 -1 4202560 2503 0 1 0 90 3 0 0 20 0 19 0 338607805 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803257045432 139803082092270 0 4 0 16800975 0 0 0 -1 0 0 0 0 0 0
[pid=7878/tid=7880] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7880/stat : 7880 (java) S 7876 7878 7878 0 -1 4202560 5 0 0 0 0 0 0 0 20 0 19 0 338607806 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803218536744 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7881] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7881/stat : 7881 (java) S 7876 7878 7878 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338607806 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803217483944 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=7878/tid=7882] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7882/stat : 7882 (java) S 7876 7878 7878 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338607806 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803216431656 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7878/tid=7883] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7883/stat : 7883 (java) S 7876 7878 7878 0 -1 4202560 10 0 0 0 0 0 0 0 20 0 19 0 338607806 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803215378856 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7884] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7884/stat : 7884 (java) S 7876 7878 7878 0 -1 4202560 4 0 0 0 0 0 0 0 20 0 19 0 338607806 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803214326568 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=7878/tid=7885] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7885/stat : 7885 (java) S 7876 7878 7878 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338607806 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803213273768 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7878/tid=7886] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7886/stat : 7886 (java) S 7876 7878 7878 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338607806 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803212221480 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7887] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7887/stat : 7887 (java) S 7876 7878 7878 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 19 0 338607806 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803211168680 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7888] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7888/stat : 7888 (java) S 7876 7878 7878 0 -1 4202560 73 0 0 0 0 0 0 0 20 0 19 0 338607807 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803081304296 240568547241 0 0 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7889] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7889/stat : 7889 (java) S 7876 7878 7878 0 -1 4202560 17 0 0 0 0 0 0 0 20 0 19 0 338607808 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803080250360 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7890] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7890/stat : 7890 (java) S 7876 7878 7878 0 -1 4202560 8 0 0 0 0 0 0 0 20 0 19 0 338607808 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803079197960 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7891] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7891/stat : 7891 (java) S 7876 7878 7878 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338607811 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803078146448 240568555296 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7892] ppid=7876 vsize=9624972 CPUtime=0.47 cores=0,2,4,6
/proc/7878/task/7892/stat : 7892 (java) S 7876 7878 7878 0 -1 4202560 2564 0 0 0 47 0 0 0 20 0 19 0 338607811 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803077094072 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=7878/tid=7893] ppid=7876 vsize=9624972 CPUtime=0.46 cores=0,2,4,6
/proc/7878/task/7893/stat : 7893 (java) S 7876 7878 7878 0 -1 4202560 3079 0 0 0 46 0 0 0 20 0 19 0 338607811 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803076041272 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7894] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7894/stat : 7894 (java) S 7876 7878 7878 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338607811 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803074989176 240568546364 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7895] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7895/stat : 7895 (java) S 7876 7878 7878 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 19 0 338607811 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803073936584 240568547241 0 4 0 16800975 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7878/tid=7896] ppid=7876 vsize=9624972 CPUtime=0 cores=0,2,4,6
/proc/7878/task/7896/stat : 7896 (java) S 7876 7878 7878 0 -1 4202560 7 0 0 0 0 0 0 0 20 0 19 0 338607871 9855971328 42481 33554432000 1073741824 1073778376 140736076822480 139803072881576 240568547241 0 4 0 16800975 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7733/tid=7740] ppid=7731 vsize=670796 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7740/stat : 7740 (python) S 7731 7733 2614 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 6 0 338579692 686895104 9803 33554432000 4194304 5683644 140734328141872 140272218985008 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7733/tid=7745] ppid=7731 vsize=670796 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7745/stat : 7745 (python) S 7731 7733 2614 0 -1 4202560 27 1011 0 0 0 0 0 0 20 0 6 0 338584675 686895104 9803 33554432000 4194304 5683644 140734328141872 140272208495152 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7733/tid=7770] ppid=7731 vsize=670796 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7770/stat : 7770 (python) S 7731 7733 2614 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 6 0 338589391 686895104 9803 33554432000 4194304 5683644 140734328141872 140272198005296 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=7733/tid=7844] ppid=7731 vsize=670796 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7844/stat : 7844 (python) S 7731 7733 2614 0 -1 4202560 22 1011 0 0 0 0 0 0 20 0 6 0 338598910 686895104 9803 33554432000 4194304 5683644 140734328141872 140272187515440 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=7733/tid=7877] ppid=7731 vsize=670796 CPUtime=0 cores=0,2,4,6
/proc/7733/task/7877/stat : 7877 (python) S 7731 7733 2614 0 -1 4202560 21 1011 0 0 0 0 0 0 20 0 6 0 338607803 686895104 9803 33554432000 4194304 5683644 140734328141872 140272177025584 240568555296 0 0 16781312 33554434 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 287.29
Current children cumulated vsize (KiB) 23660004

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 7733 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=254809
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=144977
# CPU time returned by wait4() is 1.39979
# while last known CPU time is 287.29
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 284.455
CPU time (s): 287.29
CPU user time (s): 285.25
CPU system time (s): 2.04
CPU usage (%): 100.997
Max. virtual memory (cumulated for all children) (KiB): 23660004

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.25481
system time used= 0.144977
maximum resident set size= 40236
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 19382
page faults= 1
swaps= 0
block input operations= 0
block output operations= 448
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2055
involuntary context switches= 18

runsolver used 1.11183 second user time and 2.30665 second system time

The end

Launcher Data 

Begin job on node146 at 2012-06-03 13:55:44
IDJOB=3742356
IDBENCH=90163
IDSOLVER=2327
FILE ID=node146/3742356-1338724544
RUNJOBID= node146-1338713700-2630
PBS_JOBID= 14636739
Free space on /tmp= 70808 MiB

SOLVER NAME= PB11: borg pb-dec-11.04.03
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K192.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742356-1338724544/watcher-3742356-1338724544 -o /tmp/evaluation-result-3742356-1338724544/solver-3742356-1338724544 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3742356-1338724544.opb -seed 1281393731 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 83a21ae30c37a390d06f7aafb55d6d81
RANDOM SEED=1281393731

node146.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        30229548 kB
Buffers:          287532 kB
Cached:           615032 kB
SwapCached:         1792 kB
Active:          1513112 kB
Inactive:         304012 kB
Active(anon):     915344 kB
Inactive(anon):     1712 kB
Active(file):     597768 kB
Inactive(file):   302300 kB
Unevictable:        7116 kB
Mlocked:            7116 kB
SwapTotal:      67108856 kB
SwapFree:       67099888 kB
Dirty:            103260 kB
Writeback:             0 kB
AnonPages:        920436 kB
Mapped:            21144 kB
Shmem:                48 kB
Slab:             682616 kB
SReclaimable:     121176 kB
SUnreclaim:       561440 kB
KernelStack:        1552 kB
PageTables:         8144 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:    1219112 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70708 MiB
End job on node146 at 2012-06-03 14:00:32