Trace number 3377407

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (MO) 799.35 798.765

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v80-c7120-S1494985128-003.UNKNOWN.cnf
MD5SUM6317a356b3e87abef9291d8243489542
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables80
Number of clauses7120
Sum of the clauses size49840
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 57120

Solver Data 

0.06/0.50	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.06/0.50	c INFO: solving HOME/instance-3377407-1305453019.cnf
0.06/0.50	c INFO: computing task features
0.06/0.55	c INFO: computed variable-clause graph statistics
0.06/0.55	c INFO: computed clause balance statistics
0.66/0.81	c INFO: computed variable balance statistics
0.75/0.91	c INFO: computed small-clause counts
0.75/0.91	c INFO: computed Horn-clause counts
0.75/0.92	c INFO: computed variable graph statistics
0.75/0.92	c INFO: collected features for HOME/instance-3377407-1305453019.cnf in 0.40 s
0.75/0.93	c INFO: running gnovelty+2@0 for 97 with 2399 remaining (b = 0.49)
100.11/100.13	c INFO: running gnovelty+2@0 for 97 with 2300 remaining (b = 0.11)
200.35/200.34	c INFO: running adaptg2wsat2009++@0 for 97 with 2200 remaining (b = 0.06)
299.73/299.55	c INFO: running adaptg2wsat2009++@0 for 97 with 2101 remaining (b = 0.04)
399.96/399.76	c INFO: running adaptg2wsat2009++@0 for 97 with 2001 remaining (b = 0.04)
500.33/499.97	c INFO: running gnovelty+2@0 for 97 with 1900 remaining (b = 0.03)
600.54/600.19	c INFO: running FH@0 for 97 with 1800 remaining (b = 0.02)
700.85/700.41	c INFO: running adaptg2wsat2009++@0 for 97 with 1700 remaining (b = 0.02)
799.23/798.64	c INFO: running adaptg2wsat2009++@0 for 97 with 1602 remaining (b = 0.02)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3377407-1305453019/watcher-3377407-1305453019 -o /tmp/evaluation-result-3377407-1305453019/solver-3377407-1305453019 -C 2400 -W 1300 -M 7680 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3377407-1305453019.cnf -seed 844024204 -budget 2400 

running on 2 cores: 0,2

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.88 3.81 3.39 5/292 16051
/proc/meminfo: memFree=18724428/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=15776 CPUtime=0 cores=0,2
/proc/16051/stat : 16051 (python) R 16049 16051 14954 0 -1 4202496 867 0 0 0 0 0 0 0 25 0 1 0 656901697 16154624 750 33554432000 4194304 5683644 140734318806656 18446744073709551615 4663568 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/16051/statm: 3944 751 343 364 0 431 0

[startup+0.041367 s]
/proc/loadavg: 3.88 3.81 3.39 5/292 16051
/proc/meminfo: memFree=18724428/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=18884 CPUtime=0.02 cores=0,2
/proc/16051/stat : 16051 (python) R 16049 16051 14954 0 -1 4202496 1287 0 0 0 2 0 0 0 25 0 1 0 656901697 19337216 1141 33554432000 4194304 5683644 140734318806656 18446744073709551615 4865447 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/16051/statm: 4721 1141 421 364 0 690 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 18884

[startup+0.100433 s]
/proc/loadavg: 3.88 3.81 3.39 5/292 16051
/proc/meminfo: memFree=18724428/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=101576 CPUtime=0.06 cores=0,2
/proc/16051/stat : 16051 (python) S 16049 16051 14954 0 -1 4202496 2895 0 0 0 5 1 0 0 25 0 1 0 656901697 104013824 2593 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089696256 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/16051/statm: 25394 2593 837 364 0 1694 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 101576

[startup+0.606267 s]
/proc/loadavg: 3.88 3.81 3.39 5/292 16051
/proc/meminfo: memFree=18724428/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=279936 CPUtime=0.47 cores=0,2
/proc/16051/stat : 16051 (python) R 16049 16051 14954 0 -1 4202496 10130 934 1 0 37 10 0 0 25 0 1 0 656901697 286654464 7383 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089357423 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/16051/statm: 69984 7383 1826 364 0 5908 0
Current children cumulated CPU time (s) 0.47
Current children cumulated vsize (KiB) 279936

[startup+0.700252 s]
/proc/loadavg: 3.88 3.81 3.39 5/292 16051
/proc/meminfo: memFree=18724428/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=279936 CPUtime=0.56 cores=0,2
/proc/16051/stat : 16051 (python) R 16049 16051 14954 0 -1 4202496 10130 934 1 0 46 10 0 0 25 0 1 0 656901697 286654464 7383 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089354389 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/16051/statm: 69984 7383 1826 364 0 5908 0
Current children cumulated CPU time (s) 0.56
Current children cumulated vsize (KiB) 279936

[startup+1.51011 s]
/proc/loadavg: 3.88 3.81 3.39 4/291 16080
/proc/meminfo: memFree=18749492/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=289036 CPUtime=0.79 cores=0,2
/proc/16051/stat : 16051 (python) S 16049 16051 14954 0 -1 4202496 10559 934 1 0 68 11 0 0 25 0 2 0 656901697 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16051/statm: 72259 7156 1869 364 0 8177 0
[pid=16061] ppid=16051 vsize=278792 CPUtime=0 cores=0,2
/proc/16061/stat : 16061 (python) S 16051 16051 14954 0 -1 4202560 735 0 0 0 0 0 0 0 25 0 1 0 656901790 285483008 5637 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089718287 0 0 16781312 514 2778658577097687039 0 0 17 2 0 0 0
/proc/16061/statm: 69698 5637 354 364 0 5616 0
[pid=16063] ppid=16061 vsize=1018084 CPUtime=0.57 cores=0,2
/proc/16063/stat : 16063 (gnovelty+2) R 16061 16063 16063 0 -1 4202496 553 0 0 0 57 0 0 0 25 0 1 0 656901791 1042518016 287 33554432000 134512640 135663704 4290518960 18446744073709551615 134525465 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16063/statm: 254521 287 113 282 0 254237 0
[pid=16051/tid=16062] ppid=16049 vsize=289036 CPUtime=0 cores=0,2
/proc/16051/task/16062/stat : 16062 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 25 0 2 0 656901791 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1.36
Current children cumulated vsize (KiB) 1585912

[startup+3.10088 s]
/proc/loadavg: 3.97 3.83 3.39 5/299 16098
/proc/meminfo: memFree=18693816/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=289036 CPUtime=0.79 cores=0,2
/proc/16051/stat : 16051 (python) S 16049 16051 14954 0 -1 4202496 10559 934 1 0 68 11 0 0 25 0 2 0 656901697 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16051/statm: 72259 7156 1869 364 0 8177 0
[pid=16061] ppid=16051 vsize=278792 CPUtime=0 cores=0,2
/proc/16061/stat : 16061 (python) S 16051 16051 14954 0 -1 4202560 735 0 0 0 0 0 0 0 18 0 1 0 656901790 285483008 5637 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089718287 0 0 16781312 514 2778658577097654529 0 0 17 2 0 0 0
/proc/16061/statm: 69698 5637 354 364 0 5616 0
[pid=16063] ppid=16061 vsize=1018084 CPUtime=2.16 cores=0,2
/proc/16063/stat : 16063 (gnovelty+2) R 16061 16063 16063 0 -1 4202496 553 0 0 0 216 0 0 0 25 0 1 0 656901791 1042518016 287 33554432000 134512640 135663704 4290518960 18446744073709551615 134525271 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16063/statm: 254521 287 113 282 0 254237 0
[pid=16051/tid=16062] ppid=16049 vsize=289036 CPUtime=0 cores=0,2
/proc/16051/task/16062/stat : 16062 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 25 0 2 0 656901791 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2.95
Current children cumulated vsize (KiB) 1585912

[startup+6.30026 s]
/proc/loadavg: 3.97 3.83 3.39 5/299 16098
/proc/meminfo: memFree=18692460/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=289036 CPUtime=0.79 cores=0,2
/proc/16051/stat : 16051 (python) S 16049 16051 14954 0 -1 4202496 10559 934 1 0 68 11 0 0 25 0 2 0 656901697 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16051/statm: 72259 7156 1869 364 0 8177 0
[pid=16061] ppid=16051 vsize=278792 CPUtime=0 cores=0,2
/proc/16061/stat : 16061 (python) S 16051 16051 14954 0 -1 4202560 975 0 0 0 0 0 0 0 18 0 1 0 656901790 285483008 5665 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089718287 0 0 16781312 514 2778658577097654529 0 0 17 2 0 0 0
/proc/16061/statm: 69698 5665 382 364 0 5616 0
[pid=16063] ppid=16061 vsize=1018084 CPUtime=5.36 cores=0,2
/proc/16063/stat : 16063 (gnovelty+2) R 16061 16063 16063 0 -1 4202496 553 0 0 0 536 0 0 0 25 0 1 0 656901791 1042518016 287 33554432000 134512640 135663704 4290518960 18446744073709551615 134545644 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16063/statm: 254521 287 113 282 0 254237 0
[pid=16051/tid=16062] ppid=16049 vsize=289036 CPUtime=0 cores=0,2
/proc/16051/task/16062/stat : 16062 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 25 0 2 0 656901791 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 6.15
Current children cumulated vsize (KiB) 1585912

[startup+12.7011 s]
/proc/loadavg: 3.98 3.83 3.40 5/299 16098
/proc/meminfo: memFree=18690676/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=289036 CPUtime=0.79 cores=0,2
/proc/16051/stat : 16051 (python) S 16049 16051 14954 0 -1 4202496 10559 934 1 0 68 11 0 0 25 0 2 0 656901697 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16051/statm: 72259 7156 1869 364 0 8177 0
[pid=16061] ppid=16051 vsize=278792 CPUtime=0.02 cores=0,2
/proc/16061/stat : 16061 (python) S 16051 16051 14954 0 -1 4202560 979 0 0 0 1 1 0 0 18 0 1 0 656901790 285483008 5665 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089718287 0 0 16781312 514 17788761131276271873 0 0 17 2 0 0 0
/proc/16061/statm: 69698 5665 382 364 0 5616 0
[pid=16063] ppid=16061 vsize=1018084 CPUtime=11.75 cores=0,2
/proc/16063/stat : 16063 (gnovelty+2) R 16061 16063 16063 0 -1 4202496 553 0 0 0 1175 0 0 0 25 0 1 0 656901791 1042518016 287 33554432000 134512640 135663704 4290518960 18446744073709551615 134545704 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16063/statm: 254521 287 113 282 0 254237 0
[pid=16051/tid=16062] ppid=16049 vsize=289036 CPUtime=0 cores=0,2
/proc/16051/task/16062/stat : 16062 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 25 0 2 0 656901791 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12.56
Current children cumulated vsize (KiB) 1585912

[startup+25.5009 s]
/proc/loadavg: 3.98 3.84 3.41 5/299 16098
/proc/meminfo: memFree=18691808/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=289036 CPUtime=0.79 cores=0,2
/proc/16051/stat : 16051 (python) S 16049 16051 14954 0 -1 4202496 10559 934 1 0 68 11 0 0 25 0 2 0 656901697 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16051/statm: 72259 7156 1869 364 0 8177 0
[pid=16061] ppid=16051 vsize=278792 CPUtime=0.04 cores=0,2
/proc/16061/stat : 16061 (python) S 16051 16051 14954 0 -1 4202560 985 0 0 0 2 2 0 0 18 0 1 0 656901790 285483008 5665 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089718287 0 0 16781312 514 17788761131276271873 0 0 17 2 0 0 0
/proc/16061/statm: 69698 5665 382 364 0 5616 0
[pid=16063] ppid=16061 vsize=1018084 CPUtime=24.55 cores=0,2
/proc/16063/stat : 16063 (gnovelty+2) R 16061 16063 16063 0 -1 4202496 553 0 0 0 2455 0 0 0 25 0 1 0 656901791 1042518016 287 33554432000 134512640 135663704 4290518960 18446744073709551615 134524518 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16063/statm: 254521 287 113 282 0 254237 0
[pid=16051/tid=16062] ppid=16049 vsize=289036 CPUtime=0 cores=0,2
/proc/16051/task/16062/stat : 16062 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 25 0 2 0 656901791 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 25.38
Current children cumulated vsize (KiB) 1585912

[startup+51.1005 s]
/proc/loadavg: 3.98 3.85 3.42 5/282 16138
/proc/meminfo: memFree=18714336/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=289036 CPUtime=0.79 cores=0,2
/proc/16051/stat : 16051 (python) S 16049 16051 14954 0 -1 4202496 10559 934 1 0 68 11 0 0 25 0 2 0 656901697 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16051/statm: 72259 7156 1869 364 0 8177 0
[pid=16061] ppid=16051 vsize=278792 CPUtime=0.08 cores=0,2
/proc/16061/stat : 16061 (python) S 16051 16051 14954 0 -1 4202560 995 0 0 0 4 4 0 0 18 0 1 0 656901790 285483008 5665 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089718287 0 0 16781312 514 17788761131276271873 0 0 17 2 0 0 0
/proc/16061/statm: 69698 5665 382 364 0 5616 0
[pid=16063] ppid=16061 vsize=1018084 CPUtime=50.15 cores=0,2
/proc/16063/stat : 16063 (gnovelty+2) R 16061 16063 16063 0 -1 4202496 553 0 0 0 5015 0 0 0 25 0 1 0 656901791 1042518016 287 33554432000 134512640 135663704 4290518960 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16063/statm: 254521 287 113 282 0 254237 0
[pid=16051/tid=16062] ppid=16049 vsize=289036 CPUtime=0 cores=0,2
/proc/16051/task/16062/stat : 16062 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 25 0 2 0 656901791 295972864 7156 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 51.02
Current children cumulated vsize (KiB) 1585912

[startup+102.3 s]
/proc/loadavg: 3.99 3.87 3.45 5/294 16151
/proc/meminfo: memFree=18699068/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=299636 CPUtime=0.81 cores=0,2
/proc/16051/stat : 16051 (python) S 16049 16051 14954 0 -1 4202496 11435 934 1 0 70 11 0 0 20 0 3 0 656901697 306827264 7234 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16051/statm: 74909 7234 1872 364 0 10824 0
[pid=16061] ppid=16051 vsize=289036 CPUtime=0.18 cores=0,2
/proc/16061/stat : 16061 (python) S 16051 16051 14954 0 -1 4202560 1155 0 0 0 10 8 0 0 19 0 2 0 656901790 295972864 5703 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16061/statm: 72259 5703 415 364 0 8177 0
[pid=16063] ppid=16061 vsize=1018084 CPUtime=99.18 cores=0,2
/proc/16063/stat : 16063 (gnovelty+2) T 16061 16063 16063 0 -1 4202496 553 0 0 0 9918 0 0 0 25 0 1 0 656901791 1042518016 287 33554432000 134512640 135663704 4290518960 18446744073709551615 134524551 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16063/statm: 254521 287 113 282 0 254237 0
[pid=16061/tid=16144] ppid=16051 vsize=289036 CPUtime=0 cores=0,2
/proc/16061/task/16144/stat : 16144 (python) S 16051 16051 14954 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 656911710 295972864 5703 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604453559650112 0 0 -1 0 0 0 0
[pid=16145] ppid=16051 vsize=289392 CPUtime=0 cores=0,2
/proc/16145/stat : 16145 (python) S 16051 16051 14954 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 656911711 296337408 5706 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089718342 0 0 16781312 514 4012535473590206721 0 0 17 0 0 0 0
/proc/16145/statm: 72348 5706 348 364 0 8263 0
[pid=16147] ppid=16145 vsize=1018084 CPUtime=2.15 cores=0,2
/proc/16147/stat : 16147 (gnovelty+2) R 16145 16147 16147 0 -1 4202496 551 0 0 0 215 0 0 0 25 0 1 0 656911711 1042518016 287 33554432000 134512640 135663704 4286623520 18446744073709551615 134525316 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16147/statm: 254521 287 113 282 0 254237 0
[pid=16051/tid=16062] ppid=16049 vsize=299636 CPUtime=0 cores=0,2
/proc/16051/task/16062/stat : 16062 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 25 0 3 0 656901791 306827264 7234 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
[pid=16051/tid=16146] ppid=16049 vsize=299636 CPUtime=0 cores=0,2
/proc/16051/task/16146/stat : 16146 (python) S 16049 16051 14954 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 3 0 656911711 306827264 7234 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 102.32
Current children cumulated vsize (KiB) 2914232

[startup+162.301 s]
/proc/loadavg: 3.99 3.89 3.48 5/298 16157
/proc/meminfo: memFree=18681664/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=299636 CPUtime=0.81 cores=0,2
/proc/16051/stat : 16051 (python) S 16049 16051 14954 0 -1 4202496 11435 934 1 0 70 11 0 0 20 0 3 0 656901697 306827264 7234 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/16051/statm: 74909 7234 1872 364 0 10824 0
[pid=16061] ppid=16051 vsize=289036 CPUtime=0.18 cores=0,2

################
# More data... #
################

/proc/16297/stat : 16297 (adaptg2wsat2009) T 16295 16297 16297 0 -1 4202496 717 0 0 0 10019 0 0 0 25 0 1 0 656941674 427065344 450 33554432000 134512640 135107411 4288681616 18446744073709551615 134531696 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16297/statm: 104264 450 59 146 0 103140 0
[pid=16295/tid=16305] ppid=16051 vsize=320388 CPUtime=0 cores=0,2
/proc/16295/task/16305/stat : 16305 (python) S 16051 16051 14954 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 656951693 328077312 5833 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604453817721152 0 0 -1 2 0 0 0
[pid=16306] ppid=16051 vsize=330828 CPUtime=0.2 cores=0,2
/proc/16306/stat : 16306 (python) S 16051 16051 14954 0 -1 4202560 1098 0 0 0 12 8 0 0 19 0 2 0 656951695 338767872 5861 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/16306/statm: 82707 5861 400 364 0 18610 0
[pid=16308] ppid=16306 vsize=1018084 CPUtime=100.01 cores=0,2
/proc/16308/stat : 16308 (gnovelty+2) T 16306 16308 16308 0 -1 4202496 552 0 0 0 10001 0 0 0 25 0 1 0 656951695 1042518016 286 33554432000 134512640 135663704 4289722816 18446744073709551615 134524461 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16308/statm: 254521 286 113 282 0 254237 0
[pid=16306/tid=16315] ppid=16051 vsize=330828 CPUtime=0 cores=0,2
/proc/16306/task/16315/stat : 16315 (python) S 16051 16051 14954 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 656961715 338767872 5861 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604453976158528 0 0 -1 2 0 0 0
[pid=16316] ppid=16051 vsize=341084 CPUtime=0.21 cores=0,2
/proc/16316/stat : 16316 (python) S 16051 16051 14954 0 -1 4202560 1096 0 0 0 11 10 0 0 19 0 2 0 656961717 349270016 5865 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/16316/statm: 85271 5865 400 364 0 21171 0
[pid=16318] ppid=16316 vsize=247228 CPUtime=100 cores=0,2
/proc/16318/stat : 16318 (FH) T 16316 16318 16318 0 -1 4202496 669 0 0 0 9999 1 0 0 25 0 1 0 656961717 253161472 405 33554432000 134512640 134936988 4290671936 18446744073709551615 134555005 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/16318/statm: 61807 405 41 104 0 61311 0
[pid=16316/tid=16335] ppid=16051 vsize=341084 CPUtime=0 cores=0,2
/proc/16316/task/16335/stat : 16335 (python) S 16051 16051 14954 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 656971737 349270016 5865 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604447821088832 0 0 -1 2 0 0 0
[pid=16336] ppid=16051 vsize=351528 CPUtime=0.2 cores=0,2
/proc/16336/stat : 16336 (python) S 16051 16051 14954 0 -1 4202560 1125 0 0 0 10 10 0 0 19 0 2 0 656971739 359964672 5922 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16336/statm: 87882 5922 406 364 0 23779 0
[pid=16338] ppid=16336 vsize=417056 CPUtime=98.2 cores=0,2
/proc/16338/stat : 16338 (adaptg2wsat2009) T 16336 16338 16338 0 -1 4202496 718 0 0 0 9820 0 0 0 25 0 1 0 656971739 427065344 449 33554432000 134512640 135107411 4287805984 18446744073709551615 134531935 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16338/statm: 104264 449 59 146 0 103140 0
[pid=16336/tid=16346] ppid=16051 vsize=351528 CPUtime=0 cores=0,2
/proc/16336/task/16346/stat : 16346 (python) S 16051 16051 14954 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 656981560 359964672 5922 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604447821087936 0 0 -1 2 0 0 0
[pid=16347] ppid=16051 vsize=361784 CPUtime=0 cores=0,2
/proc/16347/stat : 16347 (python) S 16051 16051 14954 0 -1 4202560 755 0 0 0 0 0 0 0 19 0 1 0 656981561 370466816 5863 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089718342 0 0 16781312 514 2570184300523683839 0 0 17 0 0 0 0
/proc/16347/statm: 90446 5863 344 364 0 26340 0
[pid=16349] ppid=16347 vsize=416916 CPUtime=0.06 cores=0,2
/proc/16349/stat : 16349 (adaptg2wsat2009) R 16347 16349 16349 0 -1 4202496 649 0 0 0 6 0 0 0 20 0 1 0 656981562 426921984 379 33554432000 134512640 135107411 4293125216 18446744073709551615 134523496 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16349/statm: 104229 379 49 146 0 103105 0
[pid=16051/tid=16062] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16062/stat : 16062 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 25 0 10 0 656901791 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
[pid=16051/tid=16146] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16146/stat : 16146 (python) S 16049 16051 14954 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 10 0 656911711 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
[pid=16051/tid=16165] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16165/stat : 16165 (python) S 16049 16051 14954 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 10 0 656921732 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
[pid=16051/tid=16180] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16180/stat : 16180 (python) S 16049 16051 14954 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 10 0 656931653 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
[pid=16051/tid=16296] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16296/stat : 16296 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 10 0 656941674 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
[pid=16051/tid=16307] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16307/stat : 16307 (python) S 16049 16051 14954 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 10 0 656951695 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
[pid=16051/tid=16317] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16317/stat : 16317 (python) S 16049 16051 14954 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 10 0 656961717 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
[pid=16051/tid=16337] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16337/stat : 16337 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 10 0 656971739 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
[pid=16051/tid=16348] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16348/stat : 16348 (python) S 16049 16051 14954 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 10 0 656981561 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 799.35
Current children cumulated vsize (KiB) 8652156

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+798.701 s]
/proc/loadavg: 4.16 4.04 3.75 5/347 16349
/proc/meminfo: memFree=18438412/32950928 swapFree=64721848/67111528
[pid=16051] ppid=16049 vsize=372028 CPUtime=0.91 cores=0,2
/proc/16051/stat : 16051 (python) S 16049 16051 14954 0 -1 4202496 17636 934 1 0 78 13 0 0 20 0 10 0 656901697 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/16051/statm: 93007 7417 1894 364 0 28901 0
[pid=16061] ppid=16051 vsize=289036 CPUtime=0.18 cores=0,2
/proc/16061/stat : 16061 (python) S 16051 16051 14954 0 -1 4202560 1155 0 0 0 10 8 0 0 19 0 2 0 656901790 295972864 5703 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16061/statm: 72259 5703 415 364 0 8177 0
[pid=16063] ppid=16061 vsize=1018084 CPUtime=99.18 cores=0,2
/proc/16063/stat : 16063 (gnovelty+2) T 16061 16063 16063 0 -1 4202496 553 0 0 0 9918 0 0 0 25 0 1 0 656901791 1042518016 287 33554432000 134512640 135663704 4290518960 18446744073709551615 134524551 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16063/statm: 254521 287 113 282 0 254237 0
[pid=16061/tid=16144] ppid=16051 vsize=289036 CPUtime=0 cores=0,2
/proc/16061/task/16144/stat : 16144 (python) S 16051 16051 14954 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 656911710 295972864 5703 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604453559650112 0 0 -1 0 0 0 0
[pid=16145] ppid=16051 vsize=289392 CPUtime=0.19 cores=0,2
/proc/16145/stat : 16145 (python) S 16051 16051 14954 0 -1 4202560 1082 0 0 0 10 9 0 0 19 0 2 0 656911711 296337408 5759 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/16145/statm: 72348 5759 400 364 0 8263 0
[pid=16147] ppid=16145 vsize=1018084 CPUtime=100.03 cores=0,2
/proc/16147/stat : 16147 (gnovelty+2) T 16145 16147 16147 0 -1 4202496 551 0 0 0 10003 0 0 0 25 0 1 0 656911711 1042518016 287 33554432000 134512640 135663704 4286623520 18446744073709551615 134525224 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/16147/statm: 254521 287 113 282 0 254237 0
[pid=16145/tid=16163] ppid=16051 vsize=289392 CPUtime=0 cores=0,2
/proc/16145/task/16163/stat : 16163 (python) S 16051 16051 14954 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 656921731 296337408 5759 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604453846934464 0 0 -1 0 0 0 0
[pid=16164] ppid=16051 vsize=299336 CPUtime=0.2 cores=0,2
/proc/16164/stat : 16164 (python) S 16051 16051 14954 0 -1 4202560 1085 0 0 0 12 8 0 0 19 0 2 0 656921732 306520064 5688 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16164/statm: 74834 5688 400 364 0 10746 0
[pid=16166] ppid=16164 vsize=417056 CPUtime=99.19 cores=0,2
/proc/16166/stat : 16166 (adaptg2wsat2009) T 16164 16166 16166 0 -1 4202496 718 0 0 0 9919 0 0 0 25 0 1 0 656921732 427065344 451 33554432000 134512640 135107411 4286920320 18446744073709551615 134532408 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16166/statm: 104264 451 59 146 0 103140 0
[pid=16164/tid=16178] ppid=16051 vsize=299336 CPUtime=0 cores=0,2
/proc/16164/task/16178/stat : 16178 (python) S 16051 16051 14954 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 656931651 306520064 5688 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604453814428096 0 0 -1 0 0 0 0
[pid=16179] ppid=16051 vsize=310132 CPUtime=0.2 cores=0,2
/proc/16179/stat : 16179 (python) S 16051 16051 14954 0 -1 4202560 1096 0 0 0 10 10 0 0 19 0 2 0 656931653 317575168 5807 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/16179/statm: 77533 5807 400 364 0 13442 0
[pid=16181] ppid=16179 vsize=417056 CPUtime=100 cores=0,2
/proc/16181/stat : 16181 (adaptg2wsat2009) T 16179 16181 16181 0 -1 4202496 716 0 0 0 10000 0 0 0 25 0 1 0 656931653 427065344 449 33554432000 134512640 135107411 4290423632 18446744073709551615 134532275 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/16181/statm: 104264 449 59 146 0 103140 0
[pid=16179/tid=16294] ppid=16051 vsize=310132 CPUtime=0 cores=0,2
/proc/16179/task/16294/stat : 16294 (python) S 16051 16051 14954 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 656941672 317575168 5807 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604443473381504 0 0 -1 0 0 0 0
[pid=16295] ppid=16051 vsize=320388 CPUtime=0.2 cores=0,2
/proc/16295/stat : 16295 (python) S 16051 16051 14954 0 -1 4202560 1090 0 0 0 13 7 0 0 19 0 2 0 656941674 328077312 5833 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16295/statm: 80097 5833 400 364 0 16003 0
[pid=16297] ppid=16295 vsize=417056 CPUtime=100.19 cores=0,2
/proc/16297/stat : 16297 (adaptg2wsat2009) T 16295 16297 16297 0 -1 4202496 717 0 0 0 10019 0 0 0 25 0 1 0 656941674 427065344 450 33554432000 134512640 135107411 4288681616 18446744073709551615 134531696 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16297/statm: 104264 450 59 146 0 103140 0
[pid=16295/tid=16305] ppid=16051 vsize=320388 CPUtime=0 cores=0,2
/proc/16295/task/16305/stat : 16305 (python) S 16051 16051 14954 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 656951693 328077312 5833 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604453817721152 0 0 -1 2 0 0 0
[pid=16306] ppid=16051 vsize=330828 CPUtime=0.2 cores=0,2
/proc/16306/stat : 16306 (python) S 16051 16051 14954 0 -1 4202560 1098 0 0 0 12 8 0 0 19 0 2 0 656951695 338767872 5861 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/16306/statm: 82707 5861 400 364 0 18610 0
[pid=16308] ppid=16306 vsize=1018084 CPUtime=100.01 cores=0,2
/proc/16308/stat : 16308 (gnovelty+2) T 16306 16308 16308 0 -1 4202496 552 0 0 0 10001 0 0 0 25 0 1 0 656951695 1042518016 286 33554432000 134512640 135663704 4289722816 18446744073709551615 134524461 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16308/statm: 254521 286 113 282 0 254237 0
[pid=16306/tid=16315] ppid=16051 vsize=330828 CPUtime=0 cores=0,2
/proc/16306/task/16315/stat : 16315 (python) S 16051 16051 14954 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 656961715 338767872 5861 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604453976158528 0 0 -1 2 0 0 0
[pid=16316] ppid=16051 vsize=341084 CPUtime=0.21 cores=0,2
/proc/16316/stat : 16316 (python) S 16051 16051 14954 0 -1 4202560 1096 0 0 0 11 10 0 0 19 0 2 0 656961717 349270016 5865 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/16316/statm: 85271 5865 400 364 0 21171 0
[pid=16318] ppid=16316 vsize=247228 CPUtime=100 cores=0,2
/proc/16318/stat : 16318 (FH) T 16316 16318 16318 0 -1 4202496 669 0 0 0 9999 1 0 0 25 0 1 0 656961717 253161472 405 33554432000 134512640 134936988 4290671936 18446744073709551615 134555005 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/16318/statm: 61807 405 41 104 0 61311 0
[pid=16316/tid=16335] ppid=16051 vsize=341084 CPUtime=0 cores=0,2
/proc/16316/task/16335/stat : 16335 (python) S 16051 16051 14954 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 656971737 349270016 5865 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604447821088832 0 0 -1 2 0 0 0
[pid=16336] ppid=16051 vsize=351528 CPUtime=0.2 cores=0,2
/proc/16336/stat : 16336 (python) S 16051 16051 14954 0 -1 4202560 1125 0 0 0 10 10 0 0 19 0 2 0 656971739 359964672 5922 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/16336/statm: 87882 5922 406 364 0 23779 0
[pid=16338] ppid=16336 vsize=417056 CPUtime=98.2 cores=0,2
/proc/16338/stat : 16338 (adaptg2wsat2009) T 16336 16338 16338 0 -1 4202496 718 0 0 0 9820 0 0 0 25 0 1 0 656971739 427065344 449 33554432000 134512640 135107411 4287805984 18446744073709551615 134531935 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/16338/statm: 104264 449 59 146 0 103140 0
[pid=16336/tid=16346] ppid=16051 vsize=351528 CPUtime=0 cores=0,2
/proc/16336/task/16346/stat : 16346 (python) S 16051 16051 14954 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 656981560 359964672 5922 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 514 18446604447821087936 0 0 -1 2 0 0 0
[pid=16347] ppid=16051 vsize=361784 CPUtime=0 cores=0,2
/proc/16347/stat : 16347 (python) S 16051 16051 14954 0 -1 4202560 755 0 0 0 0 0 0 0 19 0 1 0 656981561 370466816 5863 33554432000 4194304 5683644 140734318806656 18446744073709551615 248089718342 0 0 16781312 514 2570184300523683839 0 0 17 0 0 0 0
/proc/16347/statm: 90446 5863 344 364 0 26340 0
[pid=16349] ppid=16347 vsize=416916 CPUtime=0.06 cores=0,2
/proc/16349/stat : 16349 (adaptg2wsat2009) R 16347 16349 16349 0 -1 4202496 649 0 0 0 6 0 0 0 20 0 1 0 656981562 426921984 379 33554432000 134512640 135107411 4293125216 18446744073709551615 134523496 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/16349/statm: 104229 379 49 146 0 103105 0
[pid=16051/tid=16062] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16062/stat : 16062 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 25 0 10 0 656901791 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
[pid=16051/tid=16146] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16146/stat : 16146 (python) S 16049 16051 14954 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 10 0 656911711 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
[pid=16051/tid=16165] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16165/stat : 16165 (python) S 16049 16051 14954 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 10 0 656921732 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
[pid=16051/tid=16180] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16180/stat : 16180 (python) S 16049 16051 14954 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 10 0 656931653 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 2 0 0 0
[pid=16051/tid=16296] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16296/stat : 16296 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 10 0 656941674 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
[pid=16051/tid=16307] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16307/stat : 16307 (python) S 16049 16051 14954 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 10 0 656951695 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
[pid=16051/tid=16317] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16317/stat : 16317 (python) S 16049 16051 14954 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 10 0 656961717 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
[pid=16051/tid=16337] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16337/stat : 16337 (python) S 16049 16051 14954 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 10 0 656971739 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
[pid=16051/tid=16348] ppid=16049 vsize=372028 CPUtime=0 cores=0,2
/proc/16051/task/16348/stat : 16348 (python) S 16049 16051 14954 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 10 0 656981561 380956672 7417 33554432000 4194304 5683644 140734318806656 18446744073709551615 248101521841 0 0 16781312 2 18446604453559648320 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 799.35
Current children cumulated vsize (KiB) 8652156

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 16051 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=784880
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=139978
# CPU time returned by wait4() is 0.924858
# while last known CPU time is 799.35
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 798.765
CPU time (s): 799.35
CPU user time (s): 798.51
CPU system time (s): 0.84
CPU usage (%): 100.073
Max. virtual memory (cumulated for all children) (KiB): 8652156

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.78488
system time used= 0.139978
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 18576
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1136
involuntary context switches= 62

runsolver used 2.47062 second user time and 5.2522 second system time

The end

Launcher Data 

Begin job on node123 at 2011-05-15 11:50:19
IDJOB=3377407
IDBENCH=82716
IDSOLVER=1857
FILE ID=node123/3377407-1305453019
RUNJOBID= node123-1305452312-15689
PBS_JOBID= 13324867
Free space on /tmp= 73356 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k7-r89-v80-c7120-S1494985128-003.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3377407-1305453019/watcher-3377407-1305453019 -o /tmp/evaluation-result-3377407-1305453019/solver-3377407-1305453019 -C 2400 -W 1300 -M 7680  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3377407-1305453019.cnf -seed 844024204 -budget 2400

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= 6317a356b3e87abef9291d8243489542
RANDOM SEED=844024204

node123.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      18724832 kB
Buffers:        311544 kB
Cached:        4340552 kB
SwapCached:      10528 kB
Active:       11001228 kB
Inactive:      2939828 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      18724832 kB
SwapTotal:    67111528 kB
SwapFree:     64721848 kB
Dirty:           95524 kB
Writeback:           0 kB
AnonPages:     9286244 kB
Mapped:          25668 kB
Slab:           181860 kB
PageTables:      41344 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 32949380 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73316 MiB
End job on node123 at 2011-05-15 12:03:40