Trace number 3730700

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB10: pb_cplex 2010-06-29? (TO) 1800.09 555.116

General information on the benchmark

Namenormalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/
normalized-ooo.burch_dill.4.accl.ucl.opb
MD5SUM5c27e468756746c81988dfb461940da4
Bench CategoryDEC-SMALLINT (no optimisation, small integers)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark0.034993
Has Objective FunctionNO
SatisfiableNO
(Un)Satisfiability was provedYES
Best value of the objective function
Optimality of the best value was proved
Number of variables3939
Total number of constraints10412
Number of constraints which are clauses9740
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints672
Minimum length of a constraint1
Maximum length of a constraint13
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 69
Number of bits of the biggest number in a constraint 7
Biggest sum of numbers in a constraint 258
Number of bits of the biggest sum of numbers9
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing opb file format. File HOME/instance-3730700-1339246854.opb.
0.00/0.00	c Instance file HOME/instance-3730700-1339246854.opb
0.00/0.00	c File size is 340409 bytes.
0.00/0.05	c Highest Coefficient sum: 258
0.00/0.05	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.06	c ============================[ Problem Statistics ]=============================
0.00/0.06	c |                                                                             |
0.00/0.06	c |  Parsing time:          0.05         s                                      |
0.00/0.06	c |  Number Variables:      3939                                                |
0.00/0.06	c |  Number Constraints:    10412                                               |
0.00/0.06	c ===============================================================================
1800.09/555.11	c Remaining time 1799.94 sec.
1800.09/555.11	c 
1800.09/555.11	c *** INTERRUPTED ***
1800.09/555.11	s UNKNOWN
1800.09/555.11	c Total CPU time              : 1792.3 s
1800.09/555.11	c Memory used           : 349.64 MB
1800.09/555.11	c CPU time              : 1792.3 s
1800.09/555.11	c 
1800.09/555.11	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3730700-1339246854/watcher-3730700-1339246854 -o /tmp/evaluation-result-3730700-1339246854/solver-3730700-1339246854 -C 1800 -W 1900 -M 15500 run HOME/instance-3730700-1339246854.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.70 0.38 0.12 10/185 32052
/proc/meminfo: memFree=29753360/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0

[startup+0.0684651 s]
/proc/loadavg: 1.70 0.38 0.12 10/185 32052
/proc/meminfo: memFree=29753360/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.100234 s]
/proc/loadavg: 1.70 0.38 0.12 10/185 32052
/proc/meminfo: memFree=29753360/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.300132 s]
/proc/loadavg: 1.70 0.38 0.12 10/185 32052
/proc/meminfo: memFree=29753360/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.700163 s]
/proc/loadavg: 1.70 0.38 0.12 10/185 32052
/proc/meminfo: memFree=29753360/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+1.50019 s]
/proc/loadavg: 1.70 0.38 0.12 10/187 32054
/proc/meminfo: memFree=29729792/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=102932 CPUtime=1.6 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) R 32052 32052 31871 0 -1 4202496 27740 0 0 0 150 10 0 0 20 0 1 0 390811310 105402368 11353 33554432000 134512640 146736320 4287731744 4287725316 135523422 0 0 4096 16387 0 0 0 17 3 0 0 0 0 0
/proc/32053/statm: 25733 11353 879 2985 0 22667 0
Current children cumulated CPU time (s) 1.6
Current children cumulated vsize (KiB) 112080

[startup+3.10017 s]
/proc/loadavg: 1.70 0.38 0.12 17/194 32068
/proc/meminfo: memFree=29682708/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=117996 CPUtime=5.22 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) R 32052 32052 31871 0 -1 4202496 43833 0 0 0 504 18 0 0 20 0 1 0 390811310 120827904 15611 33554432000 134512640 146736320 4287731744 4287728536 144757825 0 0 4096 16387 0 0 0 17 1 0 0 0 0 0
/proc/32053/statm: 29499 15611 901 2985 0 26433 0
Current children cumulated CPU time (s) 5.22
Current children cumulated vsize (KiB) 127144

[startup+6.30015 s]
/proc/loadavg: 2.13 0.49 0.16 8/187 32068
/proc/meminfo: memFree=29697904/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=113124 CPUtime=8.41 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) R 32052 32052 31871 0 -1 4202496 43849 0 0 0 823 18 0 0 20 0 1 0 390811310 115838976 14548 33554432000 134512640 146736320 4287731744 4287725964 136017777 0 0 4096 16387 0 0 0 17 1 0 0 0 0 0
/proc/32053/statm: 28281 14548 902 2985 0 25215 0
Current children cumulated CPU time (s) 8.41
Current children cumulated vsize (KiB) 122272

[startup+12.7001 s]
/proc/loadavg: 2.12 0.51 0.17 3/187 32069
/proc/meminfo: memFree=29695948/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=189684 CPUtime=15.29 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) R 32052 32052 31871 0 -1 4202496 59388 0 0 0 1502 27 0 0 20 0 8 0 390811310 194236416 22069 33554432000 134512640 146736320 4287731744 4287724764 139023058 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/32053/statm: 47421 22069 910 2985 0 44355 0
Current children cumulated CPU time (s) 15.29
Current children cumulated vsize (KiB) 198832

[startup+25.5005 s]
/proc/loadavg: 3.50 0.88 0.29 15/194 32105
/proc/meminfo: memFree=29626500/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=230348 CPUtime=57.2 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) S 32052 32052 31871 0 -1 4202496 90614 0 0 0 5681 39 0 0 20 0 8 0 390811310 235876352 32427 33554432000 134512640 146736320 4287731744 4287728144 144580073 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/32053/statm: 57587 32427 928 2985 0 54521 0
[pid=32053/tid=32077] ppid=32052 vsize=230348 CPUtime=4.92 cores=1,3,5,7
/proc/32053/task/32077/stat : 32077 (pb_cplex) R 32052 32052 31871 0 -1 4202560 3085 0 0 0 490 2 0 0 20 0 8 0 390812557 235876352 32427 33554432000 134512640 146736320 4287731744 4074078380 135454131 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=32053/tid=32078] ppid=32052 vsize=230348 CPUtime=5.95 cores=1,3,5,7
/proc/32053/task/32078/stat : 32078 (pb_cplex) S 32052 32052 31871 0 -1 4202560 4659 0 0 0 593 2 0 0 20 0 8 0 390812557 235876352 32427 33554432000 134512640 146736320 4287731744 4084567696 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=32053/tid=32079] ppid=32052 vsize=230348 CPUtime=5.09 cores=1,3,5,7
/proc/32053/task/32079/stat : 32079 (pb_cplex) S 32052 32052 31871 0 -1 4202560 6272 0 0 0 507 2 0 0 20 0 8 0 390812557 235876352 32427 33554432000 134512640 146736320 4287731744 4095053456 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=32053/tid=32080] ppid=32052 vsize=230348 CPUtime=6.14 cores=1,3,5,7
/proc/32053/task/32080/stat : 32080 (pb_cplex) S 32052 32052 31871 0 -1 4202560 4945 0 0 0 613 1 0 0 20 0 8 0 390812557 235876352 32427 33554432000 134512640 146736320 4287731744 4105539216 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32081] ppid=32052 vsize=230348 CPUtime=5.84 cores=1,3,5,7
/proc/32053/task/32081/stat : 32081 (pb_cplex) S 32052 32052 31871 0 -1 4202560 3074 0 0 0 582 2 0 0 20 0 8 0 390812557 235876352 32427 33554432000 134512640 146736320 4287731744 4046450320 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=32053/tid=32082] ppid=32052 vsize=230348 CPUtime=4.1 cores=1,3,5,7
/proc/32053/task/32082/stat : 32082 (pb_cplex) S 32052 32052 31871 0 -1 4202560 2690 0 0 0 408 2 0 0 20 0 8 0 390812557 235876352 32427 33554432000 134512640 146736320 4287731744 4035964560 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32083] ppid=32052 vsize=230348 CPUtime=2.97 cores=1,3,5,7
/proc/32053/task/32083/stat : 32083 (pb_cplex) S 32052 32052 31871 0 -1 4202560 1694 0 0 0 297 0 0 0 20 0 8 0 390812557 235876352 32427 33554432000 134512640 146736320 4287731744 4025478800 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 57.2
Current children cumulated vsize (KiB) 239496

[startup+51.1005 s]
/proc/loadavg: 6.93 1.89 0.64 11/194 32105
/proc/meminfo: memFree=29593752/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=260912 CPUtime=148.46 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) R 32052 32052 31871 0 -1 4202496 137377 0 0 0 14773 73 0 0 20 0 8 0 390811310 267173888 40053 33554432000 134512640 146736320 4287731744 4287724604 135453295 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/32053/statm: 65228 40053 928 2985 0 62162 0
[pid=32053/tid=32077] ppid=32052 vsize=260912 CPUtime=17.03 cores=1,3,5,7
/proc/32053/task/32077/stat : 32077 (pb_cplex) R 32052 32052 31871 0 -1 4202560 8482 0 0 0 1697 6 0 0 20 0 8 0 390812557 267173888 40053 33554432000 134512640 146736320 4287731744 4074078280 139043568 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32078] ppid=32052 vsize=260912 CPUtime=17.3 cores=1,3,5,7
/proc/32053/task/32078/stat : 32078 (pb_cplex) R 32052 32052 31871 0 -1 4202560 10431 0 0 0 1724 6 0 0 20 0 8 0 390812557 267173888 40053 33554432000 134512640 146736320 4287731744 4084564160 135468524 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32079] ppid=32052 vsize=260912 CPUtime=16.39 cores=1,3,5,7
/proc/32053/task/32079/stat : 32079 (pb_cplex) R 32052 32052 31871 0 -1 4202560 10658 0 0 0 1634 5 0 0 20 0 8 0 390812557 267173888 40053 33554432000 134512640 146736320 4287731744 4095048800 136006382 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=32053/tid=32080] ppid=32052 vsize=260912 CPUtime=17.33 cores=1,3,5,7
/proc/32053/task/32080/stat : 32080 (pb_cplex) R 32052 32052 31871 0 -1 4202560 11055 0 0 0 1727 6 0 0 20 0 8 0 390812557 267173888 40053 33554432000 134512640 146736320 4287731744 4105535660 135454115 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32081] ppid=32052 vsize=260912 CPUtime=16.71 cores=1,3,5,7
/proc/32053/task/32081/stat : 32081 (pb_cplex) R 32052 32052 31871 0 -1 4202560 6153 0 0 0 1665 6 0 0 20 0 8 0 390812557 267173888 40053 33554432000 134512640 146736320 4287731744 4046446516 135450751 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32082] ppid=32052 vsize=260912 CPUtime=15.65 cores=1,3,5,7
/proc/32053/task/32082/stat : 32082 (pb_cplex) R 32052 32052 31871 0 -1 4202560 8376 0 0 0 1560 5 0 0 20 0 8 0 390812557 267173888 40053 33554432000 134512640 146736320 4287731744 4035960648 135525920 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32083] ppid=32052 vsize=260912 CPUtime=14.25 cores=1,3,5,7
/proc/32053/task/32083/stat : 32083 (pb_cplex) R 32052 32052 31871 0 -1 4202560 10489 0 0 0 1421 4 0 0 20 0 8 0 390812557 267173888 40053 33554432000 134512640 146736320 4287731744 4025475144 139043579 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 148.46
Current children cumulated vsize (KiB) 270060

[startup+102.307 s]
/proc/loadavg: 7.07 2.76 1.01 3/192 32210
/proc/meminfo: memFree=29133312/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=288828 CPUtime=249.21 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) R 32052 32052 31871 0 -1 4202496 236785 0 0 0 24805 116 0 0 20 0 8 0 390811310 295759872 45612 33554432000 134512640 146736320 4287731744 4287718924 136388886 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/32053/statm: 72207 45612 932 2985 0 69141 0
[pid=32053/tid=32077] ppid=32052 vsize=288828 CPUtime=25.65 cores=1,3,5,7
/proc/32053/task/32077/stat : 32077 (pb_cplex) S 32052 32052 31871 0 -1 4202560 10043 0 0 0 2555 10 0 0 20 0 8 0 390812557 295759872 45612 33554432000 134512640 146736320 4287731744 4074081936 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32078] ppid=32052 vsize=288828 CPUtime=25.73 cores=1,3,5,7
/proc/32053/task/32078/stat : 32078 (pb_cplex) S 32052 32052 31871 0 -1 4202560 19299 0 0 0 2563 10 0 0 20 0 8 0 390812557 295759872 45612 33554432000 134512640 146736320 4287731744 4084567696 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32079] ppid=32052 vsize=288828 CPUtime=24.75 cores=1,3,5,7
/proc/32053/task/32079/stat : 32079 (pb_cplex) S 32052 32052 31871 0 -1 4202560 21587 0 0 0 2466 9 0 0 20 0 8 0 390812557 295759872 45612 33554432000 134512640 146736320 4287731744 4095053456 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32080] ppid=32052 vsize=288828 CPUtime=26.17 cores=1,3,5,7
/proc/32053/task/32080/stat : 32080 (pb_cplex) S 32052 32052 31871 0 -1 4202560 20498 0 0 0 2605 12 0 0 20 0 8 0 390812557 295759872 45612 33554432000 134512640 146736320 4287731744 4105539216 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32081] ppid=32052 vsize=288828 CPUtime=25.08 cores=1,3,5,7
/proc/32053/task/32081/stat : 32081 (pb_cplex) S 32052 32052 31871 0 -1 4202560 14231 0 0 0 2498 10 0 0 20 0 8 0 390812557 295759872 45612 33554432000 134512640 146736320 4287731744 4046450320 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=32053/tid=32082] ppid=32052 vsize=288828 CPUtime=24.37 cores=1,3,5,7
/proc/32053/task/32082/stat : 32082 (pb_cplex) S 32052 32052 31871 0 -1 4202560 15352 0 0 0 2427 10 0 0 20 0 8 0 390812557 295759872 45612 33554432000 134512640 146736320 4287731744 4035964560 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32083] ppid=32052 vsize=288828 CPUtime=22.87 cores=1,3,5,7
/proc/32053/task/32083/stat : 32083 (pb_cplex) S 32052 32052 31871 0 -1 4202560 23222 0 0 0 2276 11 0 0 20 0 8 0 390812557 295759872 45612 33554432000 134512640 146736320 4287731744 4025478800 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 249.21
Current children cumulated vsize (KiB) 297976

[startup+162.3 s]
/proc/loadavg: 10.30 4.47 1.70 17/193 32211
/proc/meminfo: memFree=29042208/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=299304 CPUtime=433.75 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) R 32052 32052 31871 0 -1 4202496 450155 0 0 0 43176 199 0 0 20 0 8 0 390811310 306487296 47983 33554432000 134512640 146736320 4287731744 4287724456 139044538 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/32053/statm: 74826 47983 932 2985 0 71760 0
[pid=32053/tid=32077] ppid=32052 vsize=299304 CPUtime=48.02 cores=1,3,5,7

################
# More data... #
################

/proc/32053/statm: 83347 55723 932 2985 0 80281 0
[pid=32053/tid=32077] ppid=32052 vsize=333388 CPUtime=125.26 cores=1,3,5,7
/proc/32053/task/32077/stat : 32077 (pb_cplex) R 32052 32052 31871 0 -1 4202560 130645 0 0 0 12473 53 0 0 20 0 8 0 390812557 341389312 55723 33554432000 134512640 146736320 4287731744 4074078636 136400421 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32078] ppid=32052 vsize=333388 CPUtime=127.02 cores=1,3,5,7
/proc/32053/task/32078/stat : 32078 (pb_cplex) R 32052 32052 31871 0 -1 4202560 153427 0 0 0 12644 58 0 0 20 0 8 0 390812557 341389312 55723 33554432000 134512640 146736320 4287731744 4084564264 136385265 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=32053/tid=32079] ppid=32052 vsize=333388 CPUtime=125.82 cores=1,3,5,7
/proc/32053/task/32079/stat : 32079 (pb_cplex) R 32052 32052 31871 0 -1 4202560 185364 0 0 0 12520 62 0 0 20 0 8 0 390812557 341389312 55723 33554432000 134512640 146736320 4287731744 4095051752 136737462 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32080] ppid=32052 vsize=333388 CPUtime=125.48 cores=1,3,5,7
/proc/32053/task/32080/stat : 32080 (pb_cplex) R 32052 32052 31871 0 -1 4202560 167351 0 0 0 12486 62 0 0 20 0 8 0 390812557 341389312 55723 33554432000 134512640 146736320 4287731744 4105535800 136015579 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32081] ppid=32052 vsize=333388 CPUtime=123.29 cores=1,3,5,7
/proc/32053/task/32081/stat : 32081 (pb_cplex) S 32052 32052 31871 0 -1 4202560 151524 0 0 0 12270 59 0 0 20 0 8 0 390812557 341389312 55723 33554432000 134512640 146736320 4287731744 4046450320 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32082] ppid=32052 vsize=333388 CPUtime=123.02 cores=1,3,5,7
/proc/32053/task/32082/stat : 32082 (pb_cplex) R 32052 32052 31871 0 -1 4202560 103407 0 0 0 12253 49 0 0 20 0 8 0 390812557 341389312 55723 33554432000 134512640 146736320 4287731744 4035961036 136017791 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=32053/tid=32083] ppid=32052 vsize=333388 CPUtime=123.05 cores=1,3,5,7
/proc/32053/task/32083/stat : 32083 (pb_cplex) R 32052 32052 31871 0 -1 4202560 144815 0 0 0 12248 57 0 0 20 0 8 0 390812557 341389312 55723 33554432000 134512640 146736320 4287731744 4025474576 136403116 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 1056.27
Current children cumulated vsize (KiB) 342536

[startup+402.3 s]
/proc/loadavg: 11.84 8.72 4.10 3/193 32240
/proc/meminfo: memFree=28830536/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=341644 CPUtime=1268.46 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) R 32052 32052 31871 0 -1 4202496 1585045 0 0 0 126268 578 0 0 20 0 8 0 390811310 349843456 57533 33554432000 134512640 146736320 4287731744 4287725416 144757825 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/32053/statm: 85411 57533 932 2985 0 82345 0
[pid=32053/tid=32077] ppid=32052 vsize=341644 CPUtime=153.2 cores=1,3,5,7
/proc/32053/task/32077/stat : 32077 (pb_cplex) R 32052 32052 31871 0 -1 4202560 167698 0 0 0 15255 65 0 0 20 0 8 0 390812557 349843456 57533 33554432000 134512640 146736320 4287731744 4074079208 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=32053/tid=32078] ppid=32052 vsize=341644 CPUtime=153.85 cores=1,3,5,7
/proc/32053/task/32078/stat : 32078 (pb_cplex) R 32052 32052 31871 0 -1 4202560 198425 0 0 0 15310 75 0 0 20 0 8 0 390812557 349843456 57533 33554432000 134512640 146736320 4287731744 4084564968 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32079] ppid=32052 vsize=341644 CPUtime=153.14 cores=1,3,5,7
/proc/32053/task/32079/stat : 32079 (pb_cplex) R 32052 32052 31871 0 -1 4202560 230635 0 0 0 15237 77 0 0 20 0 8 0 390812557 349843456 57533 33554432000 134512640 146736320 4287731744 4095050400 139023098 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32080] ppid=32052 vsize=341644 CPUtime=152.76 cores=1,3,5,7
/proc/32053/task/32080/stat : 32080 (pb_cplex) R 32052 32052 31871 0 -1 4202560 209288 0 0 0 15200 76 0 0 20 0 8 0 390812557 349843456 57533 33554432000 134512640 146736320 4287731744 4105539236 144580332 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32081] ppid=32052 vsize=341644 CPUtime=149.15 cores=1,3,5,7
/proc/32053/task/32081/stat : 32081 (pb_cplex) S 32052 32052 31871 0 -1 4202560 182424 0 0 0 14843 72 0 0 20 0 8 0 390812557 349843456 57533 33554432000 134512640 146736320 4287731744 4046450320 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32082] ppid=32052 vsize=341644 CPUtime=148.75 cores=1,3,5,7
/proc/32053/task/32082/stat : 32082 (pb_cplex) S 32052 32052 31871 0 -1 4202560 137189 0 0 0 14815 60 0 0 20 0 8 0 390812557 349843456 57533 33554432000 134512640 146736320 4287731744 4035964560 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32083] ppid=32052 vsize=341644 CPUtime=149.35 cores=1,3,5,7
/proc/32053/task/32083/stat : 32083 (pb_cplex) S 32052 32052 31871 0 -1 4202560 185731 0 0 0 14865 70 0 0 20 0 8 0 390812557 349843456 57533 33554432000 134512640 146736320 4287731744 4025478800 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1268.46
Current children cumulated vsize (KiB) 350792

[startup+462.3 s]
/proc/loadavg: 11.82 9.22 4.55 10/193 32240
/proc/meminfo: memFree=28747772/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=353652 CPUtime=1476.24 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) R 32052 32052 31871 0 -1 4202496 1821857 0 0 0 146962 662 0 0 20 0 8 0 390811310 362139648 60092 33554432000 134512640 146736320 4287731744 4287724872 135999665 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/32053/statm: 88413 60092 932 2985 0 85347 0
[pid=32053/tid=32077] ppid=32052 vsize=353652 CPUtime=178.91 cores=1,3,5,7
/proc/32053/task/32077/stat : 32077 (pb_cplex) S 32052 32052 31871 0 -1 4202560 203733 0 0 0 17815 76 0 0 20 0 8 0 390812557 362139648 60092 33554432000 134512640 146736320 4287731744 4074081936 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32078] ppid=32052 vsize=353652 CPUtime=178.74 cores=1,3,5,7
/proc/32053/task/32078/stat : 32078 (pb_cplex) S 32052 32052 31871 0 -1 4202560 225960 0 0 0 17789 85 0 0 20 0 8 0 390812557 362139648 60092 33554432000 134512640 146736320 4287731744 4084567696 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32079] ppid=32052 vsize=353652 CPUtime=178.27 cores=1,3,5,7
/proc/32053/task/32079/stat : 32079 (pb_cplex) S 32052 32052 31871 0 -1 4202560 258970 0 0 0 17740 87 0 0 20 0 8 0 390812557 362139648 60092 33554432000 134512640 146736320 4287731744 4095053456 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32080] ppid=32052 vsize=353652 CPUtime=178.69 cores=1,3,5,7
/proc/32053/task/32080/stat : 32080 (pb_cplex) S 32052 32052 31871 0 -1 4202560 243205 0 0 0 17781 88 0 0 20 0 8 0 390812557 362139648 60092 33554432000 134512640 146736320 4287731744 4105539216 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=32053/tid=32081] ppid=32052 vsize=353652 CPUtime=176.85 cores=1,3,5,7
/proc/32053/task/32081/stat : 32081 (pb_cplex) S 32052 32052 31871 0 -1 4202560 213835 0 0 0 17601 84 0 0 20 0 8 0 390812557 362139648 60092 33554432000 134512640 146736320 4287731744 4046450320 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32082] ppid=32052 vsize=353652 CPUtime=177.47 cores=1,3,5,7
/proc/32053/task/32082/stat : 32082 (pb_cplex) S 32052 32052 31871 0 -1 4202560 171130 0 0 0 17676 71 0 0 20 0 8 0 390812557 362139648 60092 33554432000 134512640 146736320 4287731744 4035964560 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=32053/tid=32083] ppid=32052 vsize=353652 CPUtime=172.54 cores=1,3,5,7
/proc/32053/task/32083/stat : 32083 (pb_cplex) S 32052 32052 31871 0 -1 4202560 198345 0 0 0 17177 77 0 0 20 0 8 0 390812557 362139648 60092 33554432000 134512640 146736320 4287731744 4025478800 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1476.24
Current children cumulated vsize (KiB) 362800

[startup+522.3 s]
/proc/loadavg: 11.11 9.53 4.95 7/193 32240
/proc/meminfo: memFree=28677844/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=355944 CPUtime=1690.16 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) S 32052 32052 31871 0 -1 4202496 2043841 0 0 0 168273 743 0 0 20 0 8 0 390811310 364486656 60478 33554432000 134512640 146736320 4287731744 4287728144 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/32053/statm: 88986 60478 932 2985 0 85920 0
[pid=32053/tid=32077] ppid=32052 vsize=355944 CPUtime=205.57 cores=1,3,5,7
/proc/32053/task/32077/stat : 32077 (pb_cplex) R 32052 32052 31871 0 -1 4202560 238986 0 0 0 20467 90 0 0 20 0 8 0 390812557 364486656 60478 33554432000 134512640 146736320 4287731744 4074078800 136403116 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32078] ppid=32052 vsize=355944 CPUtime=207.69 cores=1,3,5,7
/proc/32053/task/32078/stat : 32078 (pb_cplex) R 32052 32052 31871 0 -1 4202560 256682 0 0 0 20672 97 0 0 20 0 8 0 390812557 364486656 60478 33554432000 134512640 146736320 4287731744 4084563784 135526398 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=32053/tid=32079] ppid=32052 vsize=355944 CPUtime=204.16 cores=1,3,5,7
/proc/32053/task/32079/stat : 32079 (pb_cplex) R 32052 32052 31871 0 -1 4202560 286337 0 0 0 20318 98 0 0 20 0 8 0 390812557 364486656 60478 33554432000 134512640 146736320 4287731744 4095050776 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=32053/tid=32080] ppid=32052 vsize=355944 CPUtime=205.33 cores=1,3,5,7
/proc/32053/task/32080/stat : 32080 (pb_cplex) R 32052 32052 31871 0 -1 4202560 278471 0 0 0 20432 101 0 0 20 0 8 0 390812557 364486656 60478 33554432000 134512640 146736320 4287731744 4105535692 136017474 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32081] ppid=32052 vsize=355944 CPUtime=203.83 cores=1,3,5,7
/proc/32053/task/32081/stat : 32081 (pb_cplex) R 32052 32052 31871 0 -1 4202560 245223 0 0 0 20288 95 0 0 20 0 8 0 390812557 364486656 60478 33554432000 134512640 146736320 4287731744 4046447232 139046528 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32082] ppid=32052 vsize=355944 CPUtime=204.31 cores=1,3,5,7
/proc/32053/task/32082/stat : 32082 (pb_cplex) R 32052 32052 31871 0 -1 4202560 202078 0 0 0 20348 83 0 0 20 0 8 0 390812557 364486656 60478 33554432000 134512640 146736320 4287731744 4035959924 135450751 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=32053/tid=32083] ppid=32052 vsize=355944 CPUtime=197.31 cores=1,3,5,7
/proc/32053/task/32083/stat : 32083 (pb_cplex) R 32052 32052 31871 0 -1 4202560 222193 0 0 0 19645 86 0 0 20 0 8 0 390812557 364486656 60478 33554432000 134512640 146736320 4287731744 4025475664 136403112 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1690.16
Current children cumulated vsize (KiB) 365092



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+555.1 s]
/proc/loadavg: 11.41 9.77 5.20 15/194 32243
/proc/meminfo: memFree=28635580/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=358032 CPUtime=1800.09 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) R 32052 32052 31871 0 -1 4202496 2158263 0 0 0 179227 782 0 0 20 0 8 0 390811310 366624768 60842 33554432000 134512640 146736320 4287731744 4287724308 139023093 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/32053/statm: 89508 60842 932 2985 0 86442 0
[pid=32053/tid=32077] ppid=32052 vsize=358032 CPUtime=219.28 cores=1,3,5,7
/proc/32053/task/32077/stat : 32077 (pb_cplex) R 32052 32052 31871 0 -1 4202560 255088 0 0 0 21832 96 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4074078412 136017757 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=32053/tid=32078] ppid=32052 vsize=358032 CPUtime=221.02 cores=1,3,5,7
/proc/32053/task/32078/stat : 32078 (pb_cplex) R 32052 32052 31871 0 -1 4202560 274081 0 0 0 21999 103 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4084564008 139044572 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32079] ppid=32052 vsize=358032 CPUtime=217.68 cores=1,3,5,7
/proc/32053/task/32079/stat : 32079 (pb_cplex) R 32052 32052 31871 0 -1 4202560 299763 0 0 0 21665 103 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4095050040 136015707 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32080] ppid=32052 vsize=358032 CPUtime=219.63 cores=1,3,5,7
/proc/32053/task/32080/stat : 32080 (pb_cplex) R 32052 32052 31871 0 -1 4202560 295493 0 0 0 21858 105 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4105536080 136403109 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=32053/tid=32081] ppid=32052 vsize=358032 CPUtime=218.04 cores=1,3,5,7
/proc/32053/task/32081/stat : 32081 (pb_cplex) R 32052 32052 31871 0 -1 4202560 259688 0 0 0 21704 100 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4046446796 136017474 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32082] ppid=32052 vsize=358032 CPUtime=217.56 cores=1,3,5,7
/proc/32053/task/32082/stat : 32082 (pb_cplex) R 32052 32052 31871 0 -1 4202560 218137 0 0 0 21667 89 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4035961424 136403112 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32083] ppid=32052 vsize=358032 CPUtime=211.12 cores=1,3,5,7
/proc/32053/task/32083/stat : 32083 (pb_cplex) R 32052 32052 31871 0 -1 4202560 239623 0 0 0 21019 93 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4025475276 136017769 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.09
Current children cumulated vsize (KiB) 367180

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+555.1 s]
/proc/loadavg: 11.41 9.77 5.20 15/194 32243
/proc/meminfo: memFree=28635580/32873648 swapFree=7112/7112
[pid=32052] ppid=32050 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/32052/stat : 32052 (run) S 32050 32052 31871 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 390811310 9367552 270 33554432000 4194304 5064252 140735328666944 140735328665856 242167211598 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/32052/statm: 2287 270 229 213 0 72 0
[pid=32053] ppid=32052 vsize=358032 CPUtime=1800.09 cores=1,3,5,7
/proc/32053/stat : 32053 (pb_cplex) R 32052 32052 31871 0 -1 4202496 2158263 0 0 0 179227 782 0 0 20 0 8 0 390811310 366624768 60842 33554432000 134512640 146736320 4287731744 4287724308 139023093 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/32053/statm: 89508 60842 932 2985 0 86442 0
[pid=32053/tid=32077] ppid=32052 vsize=358032 CPUtime=219.28 cores=1,3,5,7
/proc/32053/task/32077/stat : 32077 (pb_cplex) R 32052 32052 31871 0 -1 4202560 255088 0 0 0 21832 96 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4074078412 136017757 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=32053/tid=32078] ppid=32052 vsize=358032 CPUtime=221.02 cores=1,3,5,7
/proc/32053/task/32078/stat : 32078 (pb_cplex) R 32052 32052 31871 0 -1 4202560 274081 0 0 0 21999 103 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4084564008 139044572 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=32053/tid=32079] ppid=32052 vsize=358032 CPUtime=217.68 cores=1,3,5,7
/proc/32053/task/32079/stat : 32079 (pb_cplex) R 32052 32052 31871 0 -1 4202560 299763 0 0 0 21665 103 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4095050040 136015707 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32080] ppid=32052 vsize=358032 CPUtime=219.63 cores=1,3,5,7
/proc/32053/task/32080/stat : 32080 (pb_cplex) R 32052 32052 31871 0 -1 4202560 295493 0 0 0 21858 105 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4105536080 136403109 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=32053/tid=32081] ppid=32052 vsize=358032 CPUtime=218.04 cores=1,3,5,7
/proc/32053/task/32081/stat : 32081 (pb_cplex) R 32052 32052 31871 0 -1 4202560 259688 0 0 0 21704 100 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4046446796 136017474 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32082] ppid=32052 vsize=358032 CPUtime=217.56 cores=1,3,5,7
/proc/32053/task/32082/stat : 32082 (pb_cplex) R 32052 32052 31871 0 -1 4202560 218137 0 0 0 21667 89 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4035961424 136403112 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=32053/tid=32083] ppid=32052 vsize=358032 CPUtime=211.12 cores=1,3,5,7
/proc/32053/task/32083/stat : 32083 (pb_cplex) R 32052 32052 31871 0 -1 4202560 239623 0 0 0 21019 93 0 0 20 0 8 0 390812557 366624768 60842 33554432000 134512640 146736320 4287731744 4025475276 136017769 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.09
Current children cumulated vsize (KiB) 367180

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 32052 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=0
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=0
# CPU time returned by wait4() is 0
# while last known CPU time is 1800.09
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 555.116
CPU time (s): 1800.09
CPU user time (s): 1792.27
CPU system time (s): 7.82
CPU usage (%): 324.272
Max. virtual memory (cumulated for all children) (KiB): 380052

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0
maximum resident set size= 1080
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 355
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1
involuntary context switches= 2

runsolver used 1.45078 second user time and 3.90641 second system time

The end

Launcher Data 

Begin job on node126 at 2012-06-09 15:00:54
IDJOB=3730700
IDBENCH=1469
IDSOLVER=2325
FILE ID=node126/3730700-1339246854
RUNJOBID= node126-1339246833-31889
PBS_JOBID= 14644355
Free space on /tmp= 70932 MiB

SOLVER NAME= PB10: pb_cplex 2010-06-29
BENCH NAME= PB06//final/normalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/normalized-ooo.burch_dill.4.accl.ucl.opb
COMMAND LINE= run BENCHNAME
CONVERSION SCRIPT= PBconversionToLinear BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3730700-1339246854/watcher-3730700-1339246854 -o /tmp/evaluation-result-3730700-1339246854/solver-3730700-1339246854 -C 1800 -W 1900 -M 15500  run HOME/instance-3730700-1339246854.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5c27e468756746c81988dfb461940da4
RANDOM SEED=2018897968

node126.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        29754152 kB
Buffers:          595888 kB
Cached:          1466872 kB
SwapCached:         8292 kB
Active:          2037140 kB
Inactive:         119344 kB
Active(anon):      81284 kB
Inactive(anon):    11060 kB
Active(file):    1955856 kB
Inactive(file):   108284 kB
Unevictable:        7112 kB
Mlocked:            7112 kB
SwapTotal:      67108856 kB
SwapFree:       67095016 kB
Dirty:             33844 kB
Writeback:             0 kB
AnonPages:         89500 kB
Mapped:            12960 kB
Shmem:                 0 kB
Slab:             824004 kB
SReclaimable:     268112 kB
SUnreclaim:       555892 kB
KernelStack:        1512 kB
PageTables:         4656 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:     222728 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:     18432 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

CONVERSION COMMAND LINE= PBconversionToLinear HOME/instance-3730700-1339246854.opb

Free space on /tmp at the end= 70900 MiB
End job on node126 at 2012-06-09 15:10:11