Trace number 4119197

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
NaPS 1.02? (TO) 1800.04 1802.41

General information on the benchmark

NamePB15eval/normalized-PB15eval/OPT-BIGINT-LIN/
minlplib2-pb-0.1.0/opb/normalized-pb302035.lin.opb
MD5SUM22c9f60a6d1c227fc3896c108cc96d3f
Bench CategoryOPT-BIGINT-LIN (optimisation, big integers, linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark4211459
Best CPU time to get the best result obtained on this benchmark1800.5
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables165900
Total number of constraints330650
Number of constraints which are clauses165300
Number of constraints which are cardinality constraints (but not clauses)30
Number of constraints which are nor clauses,nor cardinality constraints165320
Minimum length of a constraint3
Maximum length of a constraint30
Number of terms in the objective function 165900
Biggest coefficient in the objective function 299990
Number of bits for the biggest coefficient in the objective function 19
Sum of the numbers in the objective function 1735590180
Number of bits of the sum of numbers in the objective function 31
Biggest number in a constraint 299990
Number of bits of the biggest number in a constraint 19
Biggest sum of numbers in a constraint 1735590180
Number of bits of the biggest sum of numbers31
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c NaPS 1.02b.

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2013 Olivier ROUSSEL

This is runsolver version 3.3.7 (svn: 2553)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4119197-1467256809/watcher-4119197-1467256809 -o /tmp/evaluation-result-4119197-1467256809/solver-4119197-1467256809 -C 1800 -W 1900 -M 15500 --bin-var /tmp/runsolver-exch-node140-1467246696-10265 --watchdog 1960 -S 80 naps-1.02 HOME/instance-4119197-1467256809.opb 

pid=11410
running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing Stack size limit: 81920 KiB
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 81920 KiB


[startup+0 s]
/proc/loadavg: 2.26 2.25 2.19 3/177 11413
/proc/meminfo: memFree=29338004/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=7764 CPUtime=0 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 577 0 0 0 0 0 0 0 20 0 1 0 6630544 7950336 452 33554432000 4194304 5796924 140736247988112 140734148830408 5183584 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 1941 453 114 392 0 1540 0

[startup+0.100169 s]
/proc/loadavg: 2.26 2.25 2.19 3/177 11413
/proc/meminfo: memFree=29338004/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=28432 CPUtime=0.09 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 5489 0 0 0 8 1 0 0 20 0 1 0 6630544 29114368 5046 33554432000 4194304 5796924 140736247988112 140734148830408 4600662 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 7108 5048 114 392 0 6707 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28432

[startup+0.200384 s]
/proc/loadavg: 2.26 2.25 2.19 3/177 11413
/proc/meminfo: memFree=29338004/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=48432 CPUtime=0.19 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 10841 0 0 0 18 1 0 0 20 0 1 0 6630544 49594368 10003 33554432000 4194304 5796924 140736247988112 140734148830408 4642172 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 12108 10007 114 392 0 11707 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 48432

[startup+0.300382 s]
/proc/loadavg: 2.26 2.25 2.19 3/177 11413
/proc/meminfo: memFree=29338004/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=57664 CPUtime=0.29 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 12618 0 0 0 27 2 0 0 20 0 1 0 6630544 59047936 11780 33554432000 4194304 5796924 140736247988112 140736247987416 5087776 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 14416 11780 116 392 0 14015 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 57664

[startup+0.700325 s]
/proc/loadavg: 2.26 2.25 2.19 3/177 11413
/proc/meminfo: memFree=29338004/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=73692 CPUtime=0.69 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 17184 0 0 0 66 3 0 0 20 0 1 0 6630544 75460608 15995 33554432000 4194304 5796924 140736247988112 140736247987416 5183584 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 18423 15995 128 392 0 18022 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 73692

[startup+1.50029 s]
/proc/loadavg: 2.26 2.25 2.19 3/178 11414
/proc/meminfo: memFree=29262780/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=92248 CPUtime=1.49 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 21824 0 0 0 145 4 0 0 20 0 1 0 6630544 94461952 21087 33554432000 4194304 5796924 140736247988112 140736247987368 5183584 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 23062 21088 128 392 0 22661 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 92248

[startup+3.10023 s]
/proc/loadavg: 2.26 2.25 2.19 3/178 11414
/proc/meminfo: memFree=29228768/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=146836 CPUtime=3.09 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 35492 0 0 0 302 7 0 0 20 0 1 0 6630544 150360064 34633 33554432000 4194304 5796924 140736247988112 140736247987272 5095032 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 36709 34633 128 392 0 36308 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 146836

[startup+6.30032 s]
/proc/loadavg: 2.24 2.25 2.19 3/178 11414
/proc/meminfo: memFree=29132260/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=218960 CPUtime=6.29 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 57535 0 0 0 618 11 0 0 20 0 1 0 6630544 224215040 52491 33554432000 4194304 5796924 140736247988112 140736247982632 5092509 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 54740 52491 143 392 0 54339 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 218960

[startup+12.7003 s]
/proc/loadavg: 2.22 2.24 2.19 3/178 11414
/proc/meminfo: memFree=28998516/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=358976 CPUtime=12.69 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 79119 0 0 0 1253 16 0 0 20 0 1 0 6630544 367591424 84417 33554432000 4194304 5796924 140736247988112 140736247985960 4606523 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 89744 84417 159 392 0 89343 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 358976

[startup+25.5004 s]
/proc/loadavg: 2.39 2.28 2.20 3/178 11414
/proc/meminfo: memFree=28950028/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=418348 CPUtime=25.49 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 80442 0 0 0 2531 18 0 0 20 0 1 0 6630544 428388352 96706 33554432000 4194304 5796924 140736247988112 140736247985400 4607258 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 104587 96706 160 392 0 104186 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 418348

[startup+51.1003 s]
/proc/loadavg: 2.26 2.26 2.19 3/178 11414
/proc/meminfo: memFree=28942408/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=418348 CPUtime=51.08 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 82175 0 0 0 5087 21 0 0 20 0 1 0 6630544 428388352 98754 33554432000 4194304 5796924 140736247988112 140736247985400 4607258 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 104587 98754 160 392 0 104186 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 418348

[startup+102.3 s]
/proc/loadavg: 2.27 2.26 2.20 3/177 11416
/proc/meminfo: memFree=28916340/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=449168 CPUtime=102.26 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 84941 0 0 0 10199 27 0 0 20 0 1 0 6630544 459948032 104965 33554432000 4194304 5796924 140736247988112 140736247985752 4606555 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 112292 104965 160 392 0 111891 0
Current children cumulated CPU time (s) 102.26
Current children cumulated vsize (KiB) 449168

[startup+162.3 s]
/proc/loadavg: 2.32 2.27 2.20 3/177 11417
/proc/meminfo: memFree=28913548/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=449168 CPUtime=162.21 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 87449 0 0 0 16188 33 0 0 20 0 1 0 6630544 459948032 106070 33554432000 4194304 5796924 140736247988112 140736247985752 4606507 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 112292 106070 160 392 0 111891 0
Current children cumulated CPU time (s) 162.21
Current children cumulated vsize (KiB) 449168

[startup+222.307 s]
/proc/loadavg: 2.21 2.25 2.19 3/177 11418
/proc/meminfo: memFree=28848452/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=580644 CPUtime=222.2 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 89559 0 0 0 22182 38 0 0 20 0 1 0 6630544 594579456 121240 33554432000 4194304 5796924 140736247988112 140736247985752 4606523 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 145161 121240 160 392 0 144760 0
Current children cumulated CPU time (s) 222.2
Current children cumulated vsize (KiB) 580644

[startup+282.3 s]
/proc/loadavg: 2.18 2.23 2.19 3/177 11419
/proc/meminfo: memFree=28908536/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=449168 CPUtime=282.18 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 91720 0 0 0 28175 43 0 0 20 0 1 0 6630544 459948032 106720 33554432000 4194304 5796924 140736247988112 140736247985576 4607201 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 112292 106720 160 392 0 111891 0
Current children cumulated CPU time (s) 282.18
Current children cumulated vsize (KiB) 449168

[startup+342.3 s]
/proc/loadavg: 2.31 2.26 2.19 3/177 11420
/proc/meminfo: memFree=28882972/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=499256 CPUtime=342.17 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 93424 0 0 0 34168 49 0 0 20 0 1 0 6630544 511238144 113165 33554432000 4194304 5796924 140736247988112 140736247985752 4606555 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 124814 113165 160 392 0 124413 0
Current children cumulated CPU time (s) 342.17
Current children cumulated vsize (KiB) 499256

[startup+402.3 s]
/proc/loadavg: 2.29 2.26 2.19 3/177 11468
/proc/meminfo: memFree=29049868/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=499256 CPUtime=402.16 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 94823 0 0 0 40162 54 0 0 20 0 1 0 6630544 511238144 115213 33554432000 4194304 5796924 140736247988112 140736247985752 4607192 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 124814 115213 160 392 0 124413 0
Current children cumulated CPU time (s) 402.16
Current children cumulated vsize (KiB) 499256

[startup+462.3 s]
/proc/loadavg: 2.29 2.26 2.19 3/177 11469
/proc/meminfo: memFree=28714512/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=499256 CPUtime=462.16 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 96225 0 0 0 46156 60 0 0 20 0 1 0 6630544 511238144 116749 33554432000 4194304 5796924 140736247988112 140736247985752 4606779 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 124814 116749 160 392 0 124413 0
Current children cumulated CPU time (s) 462.16
Current children cumulated vsize (KiB) 499256

[startup+522.3 s]
/proc/loadavg: 2.27 2.25 2.19 3/177 11470
/proc/meminfo: memFree=28722632/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=499256 CPUtime=522.15 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 97625 0 0 0 52151 64 0 0 20 0 1 0 6630544 511238144 113677 33554432000 4194304 5796924 140736247988112 140736247985752 4606789 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 124814 113677 160 392 0 124413 0
Current children cumulated CPU time (s) 522.15
Current children cumulated vsize (KiB) 499256

[startup+582.3 s]
/proc/loadavg: 2.24 2.25 2.19 3/177 11471
/proc/meminfo: memFree=28604804/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=712904 CPUtime=582.14 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 98876 0 0 0 58147 67 0 0 20 0 1 0 6630544 730013696 142005 33554432000 4194304 5796924 140736247988112 140736247985752 4606555 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 178226 142005 160 392 0 177825 0
Current children cumulated CPU time (s) 582.14
Current children cumulated vsize (KiB) 712904

################
# More data... #
################

[pid=11413] ppid=11410 vsize=580644 CPUtime=1062.06 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 108263 0 0 0 106115 91 0 0 20 0 1 0 6630544 594579456 123128 33554432000 4194304 5796924 140736247988112 140736247985752 4596221 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 145161 123128 160 392 0 144760 0
Current children cumulated CPU time (s) 1062.06
Current children cumulated vsize (KiB) 580644

[startup+1122.3 s]
/proc/loadavg: 2.19 2.23 2.18 3/177 11485
/proc/meminfo: memFree=28288616/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=712904 CPUtime=1122.05 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 108823 0 0 0 112113 92 0 0 20 0 1 0 6630544 730013696 146168 33554432000 4194304 5796924 140736247988112 140736247985752 4606514 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 178226 146168 160 392 0 177825 0
Current children cumulated CPU time (s) 1122.05
Current children cumulated vsize (KiB) 712904

[startup+1182.3 s]
/proc/loadavg: 2.12 2.20 2.18 4/177 11486
/proc/meminfo: memFree=28409868/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=499256 CPUtime=1182.05 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 108902 0 0 0 118110 95 0 0 20 0 1 0 6630544 511238144 114122 33554432000 4194304 5796924 140736247988112 140736247985576 4606771 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 124814 114122 160 392 0 124413 0
Current children cumulated CPU time (s) 1182.05
Current children cumulated vsize (KiB) 499256

[startup+1242.3 s]
/proc/loadavg: 2.26 2.22 2.18 3/178 11490
/proc/meminfo: memFree=28300052/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=580644 CPUtime=1242.04 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 109630 0 0 0 124107 97 0 0 20 0 1 0 6630544 594579456 136818 33554432000 4194304 5796924 140736247988112 140736247985752 4607194 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 145161 136818 160 392 0 144760 0
Current children cumulated CPU time (s) 1242.04
Current children cumulated vsize (KiB) 580644

[startup+1302.3 s]
/proc/loadavg: 2.23 2.21 2.18 3/176 11491
/proc/meminfo: memFree=28211160/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=712904 CPUtime=1302.02 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 110650 0 0 0 130103 99 0 0 20 0 1 0 6630544 730013696 157251 33554432000 4194304 5796924 140736247988112 140736247985752 4606807 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 178226 157251 160 392 0 177825 0
Current children cumulated CPU time (s) 1302.02
Current children cumulated vsize (KiB) 712904

[startup+1362.3 s]
/proc/loadavg: 2.25 2.21 2.18 3/177 11493
/proc/meminfo: memFree=28334628/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=580644 CPUtime=1362.01 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 111420 0 0 0 136096 105 0 0 20 0 1 0 6630544 594579456 129138 33554432000 4194304 5796924 140736247988112 140736247985752 4606507 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 145161 129138 160 392 0 144760 0
Current children cumulated CPU time (s) 1362.01
Current children cumulated vsize (KiB) 580644

[startup+1422.3 s]
/proc/loadavg: 2.34 2.23 2.19 3/177 11494
/proc/meminfo: memFree=28247932/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=712904 CPUtime=1422 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 112457 0 0 0 142092 108 0 0 20 0 1 0 6630544 730013696 148035 33554432000 4194304 5796924 140736247988112 140736247985752 4606768 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 178226 148035 160 392 0 177825 0
Current children cumulated CPU time (s) 1422
Current children cumulated vsize (KiB) 712904

[startup+1482.3 s]
/proc/loadavg: 2.23 2.22 2.18 4/177 11495
/proc/meminfo: memFree=28389724/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=499256 CPUtime=1481.97 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 112577 0 0 0 148086 111 0 0 20 0 1 0 6630544 511238144 114634 33554432000 4194304 5796924 140736247988112 140736247985576 4606555 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 124814 114634 160 392 0 124413 0
Current children cumulated CPU time (s) 1481.97
Current children cumulated vsize (KiB) 499256

[startup+1542.31 s]
/proc/loadavg: 2.24 2.22 2.18 3/177 11496
/proc/meminfo: memFree=28247836/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=638736 CPUtime=1541.96 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 139110 0 0 0 154079 117 0 0 20 0 1 0 6630544 654065664 148247 33554432000 4194304 5796924 140736247988112 140736247982696 5087501 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 159684 148247 171 392 0 159283 0
Current children cumulated CPU time (s) 1541.96
Current children cumulated vsize (KiB) 638736

[startup+1602.3 s]
/proc/loadavg: 2.26 2.23 2.18 3/176 11497
/proc/meminfo: memFree=28237920/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=639812 CPUtime=1601.95 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 139357 0 0 0 160078 117 0 0 20 0 1 0 6630544 655167488 148494 33554432000 4194304 5796924 140736247988112 140736247982696 4806859 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 159953 148494 171 392 0 159552 0
Current children cumulated CPU time (s) 1601.95
Current children cumulated vsize (KiB) 639812

[startup+1662.3 s]
/proc/loadavg: 2.16 2.21 2.18 3/177 11499
/proc/meminfo: memFree=28249128/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=639812 CPUtime=1661.94 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 139357 0 0 0 166077 117 0 0 20 0 1 0 6630544 655167488 148494 33554432000 4194304 5796924 140736247988112 140736247982696 5092416 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 159953 148494 171 392 0 159552 0
Current children cumulated CPU time (s) 1661.94
Current children cumulated vsize (KiB) 639812

[startup+1722.3 s]
/proc/loadavg: 2.23 2.23 2.18 3/177 11500
/proc/meminfo: memFree=28238456/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=639812 CPUtime=1721.93 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 139360 0 0 0 172076 117 0 0 20 0 1 0 6630544 655167488 148497 33554432000 4194304 5796924 140736247988112 140736247982696 4295642 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 159953 148497 172 392 0 159552 0
Current children cumulated CPU time (s) 1721.93
Current children cumulated vsize (KiB) 639812

[startup+1782.3 s]
/proc/loadavg: 2.33 2.25 2.19 3/177 11501
/proc/meminfo: memFree=28252744/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=639812 CPUtime=1781.92 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 139360 0 0 0 178075 117 0 0 20 0 1 0 6630544 655167488 148497 33554432000 4194304 5796924 140736247988112 140736247982696 4865184 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 159953 148497 172 392 0 159552 0
Current children cumulated CPU time (s) 1781.92
Current children cumulated vsize (KiB) 639812



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.4 s]
/proc/loadavg: 2.23 2.23 2.18 3/178 11504
/proc/meminfo: memFree=28250796/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=639812 CPUtime=1800.02 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) R 11410 11413 10249 0 -1 4202496 139360 0 0 0 179885 117 0 0 20 0 1 0 6630544 655167488 148497 33554432000 4194304 5796924 140736247988112 140736247982696 4865554 0 0 4096 17954 0 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 159953 148497 172 392 0 159552 0
Current children cumulated CPU time (s) 1800.02
Current children cumulated vsize (KiB) 639812

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -11413
??? exit watcher main loop
??? sigkill at end of watcher main loop
??? kill 11413 sig 9
??? end of watcher thread

Solver just ended.
Dumping a history of the last processes samples

[startup+1801.6 s]
/proc/loadavg: 2.23 2.23 2.18 2/178 11504
/proc/meminfo: memFree=28250360/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=639812 CPUtime=1800.02 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) S 11410 11413 10249 0 -1 4194304 139361 0 0 0 179885 117 0 0 20 0 1 0 6630544 655167488 148498 33554432000 4194304 5796924 140736247988112 140736247981576 5190702 0 16384 4096 17954 18446744071579510697 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 159953 148498 173 392 0 159552 0
Current children cumulated CPU time (s) 1800.02
Current children cumulated vsize (KiB) 639812

[startup+1802 s]
/proc/loadavg: 2.23 2.23 2.18 2/178 11504
/proc/meminfo: memFree=28248624/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=639812 CPUtime=1800.02 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) S 11410 11413 10249 0 -1 4194304 139361 0 0 0 179885 117 0 0 20 0 1 0 6630544 655167488 148498 33554432000 4194304 5796924 140736247988112 140736247981576 5190702 0 16384 4096 17954 18446744071579510697 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 159953 148498 173 392 0 159552 0
Current children cumulated CPU time (s) 1800.02
Current children cumulated vsize (KiB) 639812

[startup+1802.4 s]
/proc/loadavg: 2.23 2.23 2.18 2/178 11504
/proc/meminfo: memFree=28248624/32872528 swapFree=1391468/1569296
[pid=11413] ppid=11410 vsize=639812 CPUtime=1800.02 cores=0,2,4,6
/proc/11413/stat : 11413 (naps-1.02) S 11410 11413 10249 0 -1 4194304 139361 0 0 0 179885 117 0 0 20 0 1 0 6630544 655167488 148498 33554432000 4194304 5796924 140736247988112 140736247981576 5190702 0 16384 4096 17954 18446744071579510697 0 0 17 2 0 0 0 0 0
/proc/11413/statm: 159953 148498 173 392 0 159552 0
Current children cumulated CPU time (s) 1800.02
Current children cumulated vsize (KiB) 639812

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1802.41
CPU time (s): 1800.04
CPU user time (s): 1798.85
CPU system time (s): 1.18582
CPU usage (%): 99.8682
Max. virtual memory (cumulated for all children) (KiB): 843132

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1798.85
system time used= 1.18582
maximum resident set size= 741648
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 139361
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 158
involuntary context switches= 1851

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.73513 second user time and 8.21175 second system time

The end

Launcher Data 

Begin job on node140 at 2016-06-30 05:20:10
IDJOB=4119197
IDBENCH=111671
IDSOLVER=2544
FILE ID=node140/4119197-1467256809
RUNJOBID= node140-1467246696-10265
PBS_JOBID= 20004483
Free space on /tmp= 71636 MiB

SOLVER NAME= NaPS 1.02
BENCH NAME= PB15eval/normalized-PB15eval/OPT-BIGINT-LIN/minlplib2-pb-0.1.0/opb/normalized-pb302035.lin.opb
COMMAND LINE= naps-1.02 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4119197-1467256809/watcher-4119197-1467256809 -o /tmp/evaluation-result-4119197-1467256809/solver-4119197-1467256809 -C 1800 -W 1900 -M 15500 --bin-var /tmp/runsolver-exch-node140-1467246696-10265 --watchdog 1960 -S 80 naps-1.02 HOME/instance-4119197-1467256809.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 22c9f60a6d1c227fc3896c108cc96d3f
RANDOM SEED=1883822412

node140.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.708
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.41
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.708
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.708
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.708
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.708
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.708
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.708
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.708
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32872528 kB
MemFree:        29338648 kB
Buffers:          117784 kB
Cached:           920040 kB
SwapCached:            0 kB
Active:           314472 kB
Inactive:         912032 kB
Active(anon):     182136 kB
Inactive(anon):     8504 kB
Active(file):     132336 kB
Inactive(file):   903528 kB
Unevictable:     1569296 kB
Mlocked:         1391468 kB
SwapTotal:      67108856 kB
SwapFree:       67108856 kB
Dirty:             17120 kB
Writeback:             0 kB
AnonPages:       1713932 kB
Mapped:            14552 kB
Shmem:               204 kB
Slab:             605496 kB
SReclaimable:      49788 kB
SUnreclaim:       555708 kB
KernelStack:        1480 kB
PageTables:         7212 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545120 kB
Committed_AS:    1871392 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346412 kB
VmallocChunk:   34341988032 kB
HardwareCorrupted:     0 kB
AnonHugePages:    235520 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7552 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71632 MiB
End job on node140 at 2016-06-30 05:50:12