Trace number 4082253

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
NaPS 1.02? (TO) 1800.07 1802.41

General information on the benchmark

Namenormalized-PB06/OPT-BIGINT/mps-v2-20-10/plato.asu.edu/
pub/fctp/normalized-mps-v2-20-10-n370a.opb
MD5SUM28d10e1d13e82e070b03ac2e5d475941
Bench CategoryOPT-BIGINT (optimisation, big integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark45168146369
Best CPU time to get the best result obtained on this benchmark1800.09
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 4978502603
Optimality of the best value was proved NO
Number of variables155000
Total number of constraints5150
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints5150
Minimum length of a constraint31
Maximum length of a constraint3000
Number of terms in the objective function 155000
Biggest coefficient in the objective function 4294967296
Number of bits for the biggest coefficient in the objective function 33
Sum of the numbers in the objective function 29637219205243
Number of bits of the sum of numbers in the objective function 45
Biggest number in a constraint 4294967296
Number of bits of the biggest number in a constraint 33
Biggest sum of numbers in a constraint 29637219205243
Number of bits of the biggest sum of numbers45
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c NaPS 1.02b.

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2013 Olivier ROUSSEL

This is runsolver version 3.3.7 (svn: 2553)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4082253-1465239866/watcher-4082253-1465239866 -o /tmp/evaluation-result-4082253-1465239866/solver-4082253-1465239866 -C 1800 -W 1900 -M 15500 --bin-var /tmp/runsolver-exch-node151-1465227048-7039 --watchdog 1960 naps-1.02 HOME/instance-4082253-1465239866.opb 

pid=8344
running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.03 2.01 2.00 3/176 8347
/proc/meminfo: memFree=31985768/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=7456 CPUtime=0 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 538 0 0 0 0 0 0 0 20 0 1 0 105917828 7634944 412 33554432000 4194304 5796924 140733379470992 140735294998968 5183584 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 1864 412 114 392 0 1463 0

[startup+0.100123 s]
/proc/loadavg: 2.03 2.01 2.00 3/176 8347
/proc/meminfo: memFree=31985768/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=25200 CPUtime=0.09 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 4913 0 0 0 9 0 0 0 20 0 1 0 105917828 25804800 4469 33554432000 4194304 5796924 140733379470992 140735294998968 5087526 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 6300 4470 114 392 0 5899 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25200

[startup+0.200334 s]
/proc/loadavg: 2.03 2.01 2.00 3/176 8347
/proc/meminfo: memFree=31985768/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=42392 CPUtime=0.19 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 8905 0 0 0 18 1 0 0 20 0 1 0 105917828 43409408 8616 33554432000 4194304 5796924 140733379470992 140735294998968 4659057 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 10598 8617 114 392 0 10197 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 42392

[startup+0.300307 s]
/proc/loadavg: 2.03 2.01 2.00 3/176 8347
/proc/meminfo: memFree=31985768/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=55736 CPUtime=0.29 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 11418 0 0 0 28 1 0 0 20 0 1 0 105917828 57073664 11090 33554432000 4194304 5796924 140733379470992 140735294998968 5092509 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 13934 11090 116 392 0 13533 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 55736

[startup+0.700307 s]
/proc/loadavg: 2.03 2.01 2.00 3/176 8347
/proc/meminfo: memFree=31985768/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=68352 CPUtime=0.69 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 14941 0 0 0 67 2 0 0 20 0 1 0 105917828 69992448 14310 33554432000 4194304 5796924 140733379470992 140733379469592 4405921 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 17088 14310 128 392 0 16687 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 68352

[startup+1.50024 s]
/proc/loadavg: 2.03 2.01 2.00 3/177 8348
/proc/meminfo: memFree=31924168/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=78020 CPUtime=1.49 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 17225 0 0 0 147 2 0 0 20 0 1 0 105917828 79892480 16594 33554432000 4194304 5796924 140733379470992 140733379468408 4806751 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 19505 16594 128 392 0 19104 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 78020

[startup+3.10023 s]
/proc/loadavg: 2.03 2.01 2.00 3/177 8348
/proc/meminfo: memFree=31905320/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=112796 CPUtime=3.09 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 25935 0 0 0 305 4 0 0 20 0 1 0 105917828 115503104 25304 33554432000 4194304 5796924 140733379470992 140733379468520 5096709 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 28199 25304 128 392 0 27798 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 112796

[startup+6.30027 s]
/proc/loadavg: 2.03 2.01 2.00 3/177 8348
/proc/meminfo: memFree=31827480/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=213708 CPUtime=6.29 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 51790 0 0 0 621 8 0 0 20 0 1 0 105917828 218836992 50459 33554432000 4194304 5796924 140733379470992 140733377148904 4316816 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 53427 50459 139 392 0 53026 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 213708

[startup+12.7003 s]
/proc/loadavg: 2.03 2.01 2.00 3/177 8348
/proc/meminfo: memFree=31702700/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=294440 CPUtime=12.69 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 68664 0 0 0 1258 11 0 0 20 0 1 0 105917828 301506560 70614 33554432000 4194304 5796924 140733379470992 140733377605832 5096576 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 73610 70614 139 392 0 73209 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 294440

[startup+25.5004 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 8348
/proc/meminfo: memFree=31639872/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=356392 CPUtime=25.49 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 75634 0 0 0 2535 14 0 0 20 0 1 0 105917828 364945408 86027 33554432000 4194304 5796924 140733379470992 140733376918456 5092915 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 89098 86027 139 392 0 88697 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 356392

[startup+51.1004 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 8394
/proc/meminfo: memFree=31560996/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=465336 CPUtime=51.08 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 87306 0 0 0 5091 17 0 0 20 0 1 0 105917828 476504064 113215 33554432000 4194304 5796924 140733379470992 140733378711304 4863664 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 116334 113215 139 392 0 115933 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 465336

[startup+102.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8395
/proc/meminfo: memFree=30825884/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=675656 CPUtime=102.28 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 107773 0 0 0 10205 23 0 0 20 0 1 0 105917828 691871744 163586 33554432000 4194304 5796924 140733379470992 140733378639880 5087606 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 168914 163586 139 392 0 168513 0
Current children cumulated CPU time (s) 102.28
Current children cumulated vsize (KiB) 675656

[startup+162.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8396
/proc/meminfo: memFree=30006752/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=963036 CPUtime=162.24 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 126909 0 0 0 16192 32 0 0 20 0 1 0 105917828 986148864 236064 33554432000 4194304 5796924 140733379470992 140733378766552 5093157 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 240759 236064 139 392 0 240358 0
Current children cumulated CPU time (s) 162.24
Current children cumulated vsize (KiB) 963036

[startup+222.307 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8397
/proc/meminfo: memFree=29908824/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1065048 CPUtime=222.22 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 135449 0 0 0 22186 36 0 0 20 0 1 0 105917828 1090609152 253254 33554432000 4194304 5796924 140733379470992 140733379467208 5092427 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 266262 253254 144 392 0 265861 0
Current children cumulated CPU time (s) 222.22
Current children cumulated vsize (KiB) 1065048

[startup+282.3 s]
/proc/loadavg: 2.06 2.01 2.00 3/177 8398
/proc/meminfo: memFree=29851780/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1066564 CPUtime=282.19 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 135845 0 0 0 28183 36 0 0 20 0 1 0 105917828 1092161536 253638 33554432000 4194304 5796924 140733379470992 140733379467208 5096673 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 266641 253638 144 392 0 266240 0
Current children cumulated CPU time (s) 282.19
Current children cumulated vsize (KiB) 1066564

[startup+342.3 s]
/proc/loadavg: 2.02 2.01 2.00 3/178 8403
/proc/meminfo: memFree=29784476/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1068092 CPUtime=342.17 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 136254 0 0 0 34181 36 0 0 20 0 1 0 105917828 1093726208 254549 33554432000 4194304 5796924 140733379470992 140733379467208 4808064 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 267023 254549 144 392 0 266622 0
Current children cumulated CPU time (s) 342.17
Current children cumulated vsize (KiB) 1068092

[startup+402.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8404
/proc/meminfo: memFree=29689352/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1069632 CPUtime=402.16 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 136643 0 0 0 40180 36 0 0 20 0 1 0 105917828 1095303168 254938 33554432000 4194304 5796924 140733379470992 140733379467208 5096576 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 267408 254938 144 392 0 267007 0
Current children cumulated CPU time (s) 402.16
Current children cumulated vsize (KiB) 1069632

[startup+462.3 s]
/proc/loadavg: 2.04 2.01 2.00 3/177 8405
/proc/meminfo: memFree=29614956/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1070916 CPUtime=462.15 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 136963 0 0 0 46179 36 0 0 20 0 1 0 105917828 1096617984 255258 33554432000 4194304 5796924 140733379470992 140733379467208 4809286 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 267729 255258 144 392 0 267328 0
Current children cumulated CPU time (s) 462.15
Current children cumulated vsize (KiB) 1070916

[startup+522.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 8406
/proc/meminfo: memFree=29552364/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1072168 CPUtime=522.14 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 137266 0 0 0 52178 36 0 0 20 0 1 0 105917828 1097900032 256072 33554432000 4194304 5796924 140733379470992 140733379467208 5092518 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 268042 256072 144 392 0 267641 0
Current children cumulated CPU time (s) 522.14
Current children cumulated vsize (KiB) 1072168

[startup+582.3 s]
/proc/loadavg: 2.01 2.00 2.00 3/177 8452
/proc/meminfo: memFree=31002672/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1073444 CPUtime=582.13 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 137592 0 0 0 58177 36 0 0 20 0 1 0 105917828 1099206656 256398 33554432000 4194304 5796924 140733379470992 140733379467208 5096609 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 268361 256398 144 392 0 267960 0
Current children cumulated CPU time (s) 582.13
Current children cumulated vsize (KiB) 1073444


################
# More data... #
################

[pid=8347] ppid=8344 vsize=1083780 CPUtime=1062.06 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 140172 0 0 0 106169 37 0 0 20 0 1 0 105917828 1109790720 260097 33554432000 4194304 5796924 140733379470992 140733379467208 5092509 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 270945 260097 144 392 0 270544 0
Current children cumulated CPU time (s) 1062.06
Current children cumulated vsize (KiB) 1083780

[startup+1122.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8466
/proc/meminfo: memFree=30681544/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1085020 CPUtime=1122.05 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 140478 0 0 0 112168 37 0 0 20 0 1 0 105917828 1111060480 260914 33554432000 4194304 5796924 140733379470992 140733379467208 5096633 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 271255 260914 144 392 0 270854 0
Current children cumulated CPU time (s) 1122.05
Current children cumulated vsize (KiB) 1085020

[startup+1182.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8467
/proc/meminfo: memFree=30647544/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1086172 CPUtime=1182.04 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 140772 0 0 0 118167 37 0 0 20 0 1 0 105917828 1112240128 261208 33554432000 4194304 5796924 140733379470992 140733379467208 5096709 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 271543 261208 144 392 0 271142 0
Current children cumulated CPU time (s) 1182.04
Current children cumulated vsize (KiB) 1086172

[startup+1242.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8469
/proc/meminfo: memFree=30643064/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1087492 CPUtime=1242.03 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 141107 0 0 0 124166 37 0 0 20 0 1 0 105917828 1113591808 261543 33554432000 4194304 5796924 140733379470992 140733379467208 5087529 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 271873 261543 144 392 0 271472 0
Current children cumulated CPU time (s) 1242.03
Current children cumulated vsize (KiB) 1087492

[startup+1302.3 s]
/proc/loadavg: 2.08 2.02 2.01 3/177 8470
/proc/meminfo: memFree=30639904/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1088616 CPUtime=1302.02 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 141391 0 0 0 130165 37 0 0 20 0 1 0 105917828 1114742784 262338 33554432000 4194304 5796924 140733379470992 140733379467208 4865437 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 272154 262338 144 392 0 271753 0
Current children cumulated CPU time (s) 1302.02
Current children cumulated vsize (KiB) 1088616

[startup+1362.3 s]
/proc/loadavg: 2.03 2.01 2.00 3/177 8471
/proc/meminfo: memFree=30638668/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1090036 CPUtime=1362.01 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 141717 0 0 0 136164 37 0 0 20 0 1 0 105917828 1116196864 262664 33554432000 4194304 5796924 140733379470992 140733379467208 5092509 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 272509 262664 144 392 0 272108 0
Current children cumulated CPU time (s) 1362.01
Current children cumulated vsize (KiB) 1090036

[startup+1422.3 s]
/proc/loadavg: 2.01 2.00 2.00 3/177 8472
/proc/meminfo: memFree=30638984/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1091364 CPUtime=1422.01 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 142048 0 0 0 142164 37 0 0 20 0 1 0 105917828 1117556736 262995 33554432000 4194304 5796924 140733379470992 140733379467208 5100093 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 272841 262995 144 392 0 272440 0
Current children cumulated CPU time (s) 1422.01
Current children cumulated vsize (KiB) 1091364

[startup+1482.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8473
/proc/meminfo: memFree=30670764/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1092668 CPUtime=1482 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 142372 0 0 0 148162 38 0 0 20 0 1 0 105917828 1118892032 263319 33554432000 4194304 5796924 140733379470992 140733379467208 5096553 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 273167 263319 144 392 0 272766 0
Current children cumulated CPU time (s) 1482
Current children cumulated vsize (KiB) 1092668

[startup+1542.31 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 8478
/proc/meminfo: memFree=30620196/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1093964 CPUtime=1542 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 142714 0 0 0 154162 38 0 0 20 0 1 0 105917828 1120219136 264172 33554432000 4194304 5796924 140733379470992 140733379467208 5092526 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 273491 264172 144 392 0 273090 0
Current children cumulated CPU time (s) 1542
Current children cumulated vsize (KiB) 1093964

[startup+1602.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8479
/proc/meminfo: memFree=30621048/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1095336 CPUtime=1601.98 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 143045 0 0 0 160160 38 0 0 20 0 1 0 105917828 1121624064 264503 33554432000 4194304 5796924 140733379470992 140733379464712 4865437 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 273834 264503 144 392 0 273433 0
Current children cumulated CPU time (s) 1601.98
Current children cumulated vsize (KiB) 1095336

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8480
/proc/meminfo: memFree=30620992/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1097780 CPUtime=1661.97 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 143657 0 0 0 166159 38 0 0 20 0 1 0 105917828 1124126720 265626 33554432000 4194304 5796924 140733379470992 140733379467208 5096709 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 274445 265626 144 392 0 274044 0
Current children cumulated CPU time (s) 1661.97
Current children cumulated vsize (KiB) 1097780

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8481
/proc/meminfo: memFree=30623496/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1099144 CPUtime=1721.96 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 144001 0 0 0 172158 38 0 0 20 0 1 0 105917828 1125523456 265970 33554432000 4194304 5796924 140733379470992 140733379464712 4865437 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 274786 265970 144 392 0 274385 0
Current children cumulated CPU time (s) 1721.96
Current children cumulated vsize (KiB) 1099144

[startup+1782.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 8482
/proc/meminfo: memFree=30606868/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1100428 CPUtime=1781.96 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 144346 0 0 0 178157 39 0 0 20 0 1 0 105917828 1126838272 266315 33554432000 4194304 5796924 140733379470992 140733379467208 5087772 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 275107 266315 144 392 0 274706 0
Current children cumulated CPU time (s) 1781.96
Current children cumulated vsize (KiB) 1100428



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.4 s]
/proc/loadavg: 2.06 2.02 2.00 3/177 8482
/proc/meminfo: memFree=30623452/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1100860 CPUtime=1800.05 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) R 8344 8347 7022 0 -1 4202496 144445 0 0 0 179966 39 0 0 20 0 1 0 105917828 1127280640 266414 33554432000 4194304 5796924 140733379470992 140733379467208 4316816 0 0 4096 17954 0 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 275215 266414 144 392 0 274814 0
Current children cumulated CPU time (s) 1800.05
Current children cumulated vsize (KiB) 1100860

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -8347
??? exit watcher main loop
??? sigkill at end of watcher main loop
??? kill 8347 sig 9
??? end of watcher thread

Solver just ended.
Dumping a history of the last processes samples

[startup+1801.6 s]
/proc/loadavg: 2.06 2.02 2.00 2/177 8482
/proc/meminfo: memFree=30623452/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1100860 CPUtime=1800.05 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) S 8344 8347 7022 0 -1 4194304 144446 0 0 0 179966 39 0 0 20 0 1 0 105917828 1127280640 266415 33554432000 4194304 5796924 140733379470992 140733379463688 5190702 0 16384 4096 17954 18446744071579510697 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 275215 266415 145 392 0 274814 0
Current children cumulated CPU time (s) 1800.05
Current children cumulated vsize (KiB) 1100860

[startup+1802 s]
/proc/loadavg: 2.06 2.02 2.00 2/177 8482
/proc/meminfo: memFree=30623452/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1100860 CPUtime=1800.05 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) S 8344 8347 7022 0 -1 4194304 144446 0 0 0 179966 39 0 0 20 0 1 0 105917828 1127280640 266415 33554432000 4194304 5796924 140733379470992 140733379463688 5190702 0 16384 4096 17954 18446744071579510697 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 275215 266415 145 392 0 274814 0
Current children cumulated CPU time (s) 1800.05
Current children cumulated vsize (KiB) 1100860

[startup+1802.4 s]
/proc/loadavg: 2.06 2.02 2.00 2/177 8482
/proc/meminfo: memFree=30623452/32872528 swapFree=9100/9100
[pid=8347] ppid=8344 vsize=1100860 CPUtime=1800.05 cores=1,3,5,7
/proc/8347/stat : 8347 (naps-1.02) S 8344 8347 7022 0 -1 4194304 144446 0 0 0 179966 39 0 0 20 0 1 0 105917828 1127280640 266415 33554432000 4194304 5796924 140733379470992 140733379463688 5190702 0 16384 4096 17954 18446744071579510697 0 0 17 3 0 0 0 0 0
/proc/8347/statm: 275215 266415 145 392 0 274814 0
Current children cumulated CPU time (s) 1800.05
Current children cumulated vsize (KiB) 1100860

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1802.41
CPU time (s): 1800.07
CPU user time (s): 1799.67
CPU system time (s): 0.395939
CPU usage (%): 99.8701
Max. virtual memory (cumulated for all children) (KiB): 1131764

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1799.67
system time used= 0.395939
maximum resident set size= 1108888
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 144446
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 107
involuntary context switches= 1952

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.09507 second user time and 6.23105 second system time

The end

Launcher Data 

Begin job on node151 at 2016-06-06 21:04:27
IDJOB=4082253
IDBENCH=2284
IDSOLVER=2544
FILE ID=node151/4082253-1465239866
RUNJOBID= node151-1465227048-7039
PBS_JOBID= 19843240
Free space on /tmp= 71736 MiB

SOLVER NAME= NaPS 1.02
BENCH NAME= PB06//final/normalized-PB06/OPT-BIGINT/mps-v2-20-10/plato.asu.edu/pub/fctp/normalized-mps-v2-20-10-n370a.opb
COMMAND LINE= naps-1.02 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4082253-1465239866/watcher-4082253-1465239866 -o /tmp/evaluation-result-4082253-1465239866/solver-4082253-1465239866 -C 1800 -W 1900 -M 15500 --bin-var /tmp/runsolver-exch-node151-1465227048-7039 --watchdog 1960  naps-1.02 HOME/instance-4082253-1465239866.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 28d10e1d13e82e070b03ac2e5d475941
RANDOM SEED=82203342

node151.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.334
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.334
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.334
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.334
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.334
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.334
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.334
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.334
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32872528 kB
MemFree:        31985924 kB
Buffers:           17292 kB
Cached:            92864 kB
SwapCached:         1732 kB
Active:           134060 kB
Inactive:          42892 kB
Active(anon):      68908 kB
Inactive(anon):      604 kB
Active(file):      65152 kB
Inactive(file):    42288 kB
Unevictable:        9100 kB
Mlocked:            9100 kB
SwapTotal:      67108856 kB
SwapFree:       67098616 kB
Dirty:             12496 kB
Writeback:             0 kB
AnonPages:         79508 kB
Mapped:             7940 kB
Shmem:                 0 kB
Slab:             566520 kB
SReclaimable:      10200 kB
SUnreclaim:       556320 kB
KernelStack:        1464 kB
PageTables:         3644 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545120 kB
Committed_AS:     200992 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346412 kB
VmallocChunk:   34341988032 kB
HardwareCorrupted:     0 kB
AnonHugePages:     57344 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7552 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71732 MiB
End job on node151 at 2016-06-06 21:34:29