Trace number 3377935

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-industrial (fixed)? (TO) 1302.43 1300.03

General information on the benchmark

Nameapplication/kullmann/AES/Benchmarks/
64/aes_64_1_keyfind_1.cnf
MD5SUM9e18e0ee8a56ea2c59fe8e682ff6c747
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark32.783
Satisfiable
(Un)Satisfiability was proved
Number of variables596
Number of clauses2780
Sum of the clauses size10460
Maximum clause length5
Minimum clause length1
Number of clauses of size 1132
Number of clauses of size 280
Number of clauses of size 31168
Number of clauses of size 4336
Number of clauses of size 51064
Number of clauses of size over 50

Solver Data 

0.56/1.07	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.56/1.07	c INFO: solving HOME/instance-3377935-1305456790.cnf
0.56/1.07	c INFO: computing task features
0.56/1.08	c INFO: computed variable-clause graph statistics
0.56/1.08	c INFO: computed clause balance statistics
0.63/1.14	c INFO: computed variable balance statistics
0.63/1.18	c INFO: computed small-clause counts
0.63/1.18	c INFO: computed Horn-clause counts
0.63/1.19	c INFO: computed variable graph statistics
0.63/1.19	c INFO: collected features for HOME/instance-3377935-1305456790.cnf in 0.11 s
0.72/1.20	c INFO: running lingeling-276@0 for 582 with 2399 remaining (b = 0.58)
584.17/583.44	c INFO: running lingeling-276@582 for 291 with 1817 remaining (b = 0.17)
876.42/875.05	c INFO: running lingeling-276@0 for 194 with 1526 remaining (b = 0.35)
1072.24/1070.49	c INFO: running cryptominisat-2.9.0@0 for 485 with 1330 remaining (b = 0.41)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3377935-1305456790/watcher-3377935-1305456790 -o /tmp/evaluation-result-3377935-1305456790/solver-3377935-1305456790 -C 2400 -W 1300 -M 7680 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3377935-1305456790.cnf -seed 1792784160 -budget 2400 

running on 2 cores: 5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.54 3.15 3.47 4/300 17186
/proc/meminfo: memFree=15912808/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=15388 CPUtime=0 cores=5,7
/proc/17185/stat : 17185 (python) R 17181 17185 15311 0 -1 4202496 813 0 0 0 0 0 0 0 17 0 1 0 657281812 15757312 699 33554432000 4194304 5683644 140733975272880 18446744073709551615 5116815 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/17185/statm: 3847 699 334 364 0 334 0

[startup+0.021303 s]
/proc/loadavg: 2.54 3.15 3.47 4/300 17186
/proc/meminfo: memFree=15912808/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=16240 CPUtime=0.01 cores=5,7
/proc/17185/stat : 17185 (python) R 17181 17185 15311 0 -1 4202496 1050 0 0 0 1 0 0 0 17 0 1 0 657281812 16629760 922 33554432000 4194304 5683644 140733975272880 18446744073709551615 4539653 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/17185/statm: 4060 922 381 364 0 547 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16240

[startup+0.100313 s]
/proc/loadavg: 2.54 3.15 3.47 4/300 17186
/proc/meminfo: memFree=15912808/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=34424 CPUtime=0.06 cores=5,7
/proc/17185/stat : 17185 (python) R 17181 17185 15311 0 -1 4202496 1552 0 0 0 4 2 0 0 17 0 1 0 657281812 35250176 1388 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370955264 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/17185/statm: 8606 1388 448 364 0 944 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 34424

[startup+0.300267 s]
/proc/loadavg: 2.54 3.15 3.47 4/300 17186
/proc/meminfo: memFree=15912808/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=121292 CPUtime=0.16 cores=5,7
/proc/17185/stat : 17185 (python) R 17181 17185 15311 0 -1 4202496 3947 0 1 0 8 8 0 0 17 0 1 0 657281812 124203008 3577 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370955264 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/17185/statm: 30323 3577 970 364 0 2653 0
Current children cumulated CPU time (s) 0.16
Current children cumulated vsize (KiB) 121292

[startup+0.700207 s]
/proc/loadavg: 2.54 3.15 3.47 4/300 17186
/proc/meminfo: memFree=15912808/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=135752 CPUtime=0.37 cores=5,7
/proc/17185/stat : 17185 (python) R 17181 17185 15311 0 -1 4202496 6018 931 1 0 16 21 0 0 16 0 1 0 657281812 139010048 4170 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370953221 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/17185/statm: 33938 4170 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.37
Current children cumulated vsize (KiB) 135752

[startup+1.50104 s]
/proc/loadavg: 2.54 3.15 3.47 5/312 17214
/proc/meminfo: memFree=15830980/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=288880 CPUtime=0.72 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 9798 931 1 0 42 30 0 0 17 0 2 0 657281812 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 72220 7115 1875 364 0 8138 0
Current children cumulated CPU time (s) 0.72
Current children cumulated vsize (KiB) 288880

[startup+3.10075 s]
/proc/loadavg: 2.66 3.16 3.47 6/324 17226
/proc/meminfo: memFree=15799008/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=288880 CPUtime=0.72 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 9798 931 1 0 42 30 0 0 17 0 2 0 657281812 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 72220 7115 1875 364 0 8138 0
[pid=17218] ppid=17185 vsize=278636 CPUtime=0 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 740 0 0 0 0 0 0 0 18 0 1 0 657281932 285323264 5590 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977295 0 0 16781312 514 3509324735685066751 0 0 17 7 0 0 0
/proc/17218/statm: 69659 5590 354 364 0 5577 0
[pid=17220] ppid=17218 vsize=6708 CPUtime=1.89 cores=5,7
/proc/17220/stat : 17220 (lingeling) R 17218 17220 17220 0 -1 4202496 1468 0 0 0 189 0 0 0 25 0 1 0 657281932 6868992 877 33554432000 4194304 4352236 140734807614176 18446744073709551615 4328112 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/17220/statm: 1677 877 129 39 0 757 0
[pid=17185/tid=17219] ppid=17181 vsize=288880 CPUtime=0 cores=5,7
/proc/17185/task/17219/stat : 17219 (python) S 17181 17185 15311 0 -1 4202560 32 931 0 0 0 0 0 0 18 0 2 0 657281932 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2.61
Current children cumulated vsize (KiB) 574224

[startup+6.30118 s]
/proc/loadavg: 2.66 3.16 3.47 5/318 17262
/proc/meminfo: memFree=15815860/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=288880 CPUtime=0.72 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 9798 931 1 0 42 30 0 0 17 0 2 0 657281812 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 72220 7115 1875 364 0 8138 0
[pid=17218] ppid=17185 vsize=278636 CPUtime=0 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 889 0 0 0 0 0 0 0 18 0 1 0 657281932 285323264 5613 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977295 0 0 16781312 514 3509324735685034245 0 0 17 7 0 0 0
/proc/17218/statm: 69659 5613 377 364 0 5577 0
[pid=17220] ppid=17218 vsize=8508 CPUtime=5.09 cores=5,7
/proc/17220/stat : 17220 (lingeling) R 17218 17220 17220 0 -1 4202496 2238 0 0 0 509 0 0 0 25 0 1 0 657281932 8712192 1327 33554432000 4194304 4352236 140734807614176 18446744073709551615 4329086 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/17220/statm: 2127 1327 129 39 0 1207 0
[pid=17185/tid=17219] ppid=17181 vsize=288880 CPUtime=0 cores=5,7
/proc/17185/task/17219/stat : 17219 (python) S 17181 17185 15311 0 -1 4202560 32 931 0 0 0 0 0 0 18 0 2 0 657281932 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.81
Current children cumulated vsize (KiB) 576024

[startup+12.7011 s]
/proc/loadavg: 2.92 3.21 3.49 6/324 17296
/proc/meminfo: memFree=15788504/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=288880 CPUtime=0.72 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 9798 931 1 0 42 30 0 0 17 0 2 0 657281812 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 72220 7115 1875 364 0 8138 0
[pid=17218] ppid=17185 vsize=278636 CPUtime=0.02 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 892 0 0 0 1 1 0 0 18 0 1 0 657281932 285323264 5613 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977295 0 0 16781312 514 14055998319813951749 0 0 17 7 0 0 0
/proc/17218/statm: 69659 5613 377 364 0 5577 0
[pid=17220] ppid=17218 vsize=8640 CPUtime=11.49 cores=5,7
/proc/17220/stat : 17220 (lingeling) R 17218 17220 17220 0 -1 4202496 2815 0 0 0 1149 0 0 0 25 0 1 0 657281932 8847360 1360 33554432000 4194304 4352236 140734807614176 18446744073709551615 4328039 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/17220/statm: 2160 1360 129 39 0 1240 0
[pid=17185/tid=17219] ppid=17181 vsize=288880 CPUtime=0 cores=5,7
/proc/17185/task/17219/stat : 17219 (python) S 17181 17185 15311 0 -1 4202560 32 931 0 0 0 0 0 0 18 0 2 0 657281932 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.23
Current children cumulated vsize (KiB) 576156

[startup+25.5008 s]
/proc/loadavg: 3.16 3.25 3.49 6/324 17296
/proc/meminfo: memFree=15782168/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=288880 CPUtime=0.72 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 9798 931 1 0 42 30 0 0 17 0 2 0 657281812 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 72220 7115 1875 364 0 8138 0
[pid=17218] ppid=17185 vsize=278636 CPUtime=0.04 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 898 0 0 0 2 2 0 0 18 0 1 0 657281932 285323264 5613 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977295 0 0 16781312 514 14055998319813951749 0 0 17 7 0 0 0
/proc/17218/statm: 69659 5613 377 364 0 5577 0
[pid=17220] ppid=17218 vsize=10820 CPUtime=24.28 cores=5,7
/proc/17220/stat : 17220 (lingeling) R 17218 17220 17220 0 -1 4202496 4416 0 0 0 2428 0 0 0 25 0 1 0 657281932 11079680 1905 33554432000 4194304 4352236 140734807614176 18446744073709551615 4328105 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/17220/statm: 2705 1905 129 39 0 1785 0
[pid=17185/tid=17219] ppid=17181 vsize=288880 CPUtime=0 cores=5,7
/proc/17185/task/17219/stat : 17219 (python) S 17181 17185 15311 0 -1 4202560 32 931 0 0 0 0 0 0 18 0 2 0 657281932 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.04
Current children cumulated vsize (KiB) 578336

[startup+51.1002 s]
/proc/loadavg: 3.45 3.31 3.50 6/324 17297
/proc/meminfo: memFree=15776512/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=288880 CPUtime=0.72 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 9798 931 1 0 42 30 0 0 17 0 2 0 657281812 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 72220 7115 1875 364 0 8138 0
[pid=17218] ppid=17185 vsize=278636 CPUtime=0.09 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 909 0 0 0 5 4 0 0 18 0 1 0 657281932 285323264 5613 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977295 0 0 16781312 514 14055998319813951749 0 0 17 7 0 0 0
/proc/17218/statm: 69659 5613 377 364 0 5577 0
[pid=17220] ppid=17218 vsize=9668 CPUtime=49.89 cores=5,7
/proc/17220/stat : 17220 (lingeling) R 17218 17220 17220 0 -1 4202496 6624 0 0 0 4988 1 0 0 25 0 1 0 657281932 9900032 1617 33554432000 4194304 4352236 140734807614176 18446744073709551615 4328112 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/17220/statm: 2417 1617 129 39 0 1497 0
[pid=17185/tid=17219] ppid=17181 vsize=288880 CPUtime=0 cores=5,7
/proc/17185/task/17219/stat : 17219 (python) S 17181 17185 15311 0 -1 4202560 32 931 0 0 0 0 0 0 18 0 2 0 657281932 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.7
Current children cumulated vsize (KiB) 577184

[startup+102.312 s]
/proc/loadavg: 3.78 3.42 3.53 6/324 17298
/proc/meminfo: memFree=15767108/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=288880 CPUtime=0.72 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 9798 931 1 0 42 30 0 0 17 0 2 0 657281812 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 72220 7115 1875 364 0 8138 0
[pid=17218] ppid=17185 vsize=278636 CPUtime=0.21 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 934 0 0 0 13 8 0 0 18 0 1 0 657281932 285323264 5613 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977295 0 0 16781312 514 14055998319813951749 0 0 17 7 0 0 0
/proc/17218/statm: 69659 5613 377 364 0 5577 0
[pid=17220] ppid=17218 vsize=13004 CPUtime=101.1 cores=5,7
/proc/17220/stat : 17220 (lingeling) R 17218 17220 17220 0 -1 4202496 10274 0 0 0 10109 1 0 0 25 0 1 0 657281932 13316096 2451 33554432000 4194304 4352236 140734807614176 18446744073709551615 4207882 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/17220/statm: 3251 2451 129 39 0 2331 0
[pid=17185/tid=17219] ppid=17181 vsize=288880 CPUtime=0 cores=5,7
/proc/17185/task/17219/stat : 17219 (python) S 17181 17185 15311 0 -1 4202560 32 931 0 0 0 0 0 0 18 0 2 0 657281932 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 102.03
Current children cumulated vsize (KiB) 580520

[startup+162.301 s]
/proc/loadavg: 3.91 3.52 3.56 6/324 17300
/proc/meminfo: memFree=15762728/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=288880 CPUtime=0.72 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 9798 931 1 0 42 30 0 0 17 0 2 0 657281812 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 72220 7115 1875 364 0 8138 0
[pid=17218] ppid=17185 vsize=278636 CPUtime=0.35 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 952 0 0 0 20 15 0 0 18 0 1 0 657281932 285323264 5613 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977295 0 0 16781312 514 14055998319813951749 0 0 17 7 0 0 0
/proc/17218/statm: 69659 5613 377 364 0 5577 0
[pid=17220] ppid=17218 vsize=14672 CPUtime=161.08 cores=5,7
/proc/17220/stat : 17220 (lingeling) R 17218 17220 17220 0 -1 4202496 16003 0 0 0 16107 1 0 0 25 0 1 0 657281932 15024128 2868 33554432000 4194304 4352236 140734807614176 18446744073709551615 4328009 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/17220/statm: 3668 2868 129 39 0 2748 0
[pid=17185/tid=17219] ppid=17181 vsize=288880 CPUtime=0 cores=5,7
/proc/17185/task/17219/stat : 17219 (python) S 17181 17185 15311 0 -1 4202560 32 931 0 0 0 0 0 0 18 0 2 0 657281932 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 162.15
Current children cumulated vsize (KiB) 582188

[startup+222.301 s]
/proc/loadavg: 3.97 3.61 3.58 6/324 17301
/proc/meminfo: memFree=15756484/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=288880 CPUtime=0.72 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 9798 931 1 0 42 30 0 0 17 0 2 0 657281812 295813120 7115 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 72220 7115 1875 364 0 8138 0
[pid=17218] ppid=17185 vsize=278636 CPUtime=0.48 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 973 0 0 0 27 21 0 0 18 0 1 0 657281932 285323264 5613 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977295 0 0 16781312 514 14055998319813951749 0 0 17 7 0 0 0

################
# More data... #
################

[startup+1122.3 s]
/proc/loadavg: 4.04 4.05 3.87 6/356 17369
/proc/meminfo: memFree=15495144/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=309764 CPUtime=0.75 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 11372 931 1 0 44 31 0 0 19 0 4 0 657281812 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 77441 7210 1880 364 0 13353 0
[pid=17218] ppid=17185 vsize=288880 CPUtime=1.92 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 1204 0 0 0 101 91 0 0 18 0 2 0 657281932 295813120 5658 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/17218/statm: 72220 5658 417 364 0 8138 0
[pid=17220] ppid=17218 vsize=31572 CPUtime=873.82 cores=5,7
/proc/17220/stat : 17220 (lingeling) T 17218 17220 17220 0 -1 4202496 72292 0 0 0 87373 9 0 0 25 0 1 0 657281932 32329728 7093 33554432000 4194304 4352236 140734807614176 18446744073709551615 4328194 0 0 16781312 17506 18446744071562691360 0 0 17 5 0 0 0
/proc/17220/statm: 7893 7093 129 39 0 6973 0
[pid=17218/tid=17321] ppid=17185 vsize=288880 CPUtime=0 cores=5,7
/proc/17218/task/17321/stat : 17321 (python) S 17185 17185 15311 0 -1 4202560 43 0 0 0 0 0 0 0 18 0 2 0 657340155 295813120 5658 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 514 18446604462319706176 0 0 -1 5 0 0 0
[pid=17336] ppid=17185 vsize=289264 CPUtime=0.45 cores=5,7
/proc/17336/stat : 17336 (python) S 17185 17185 15311 0 -1 4202560 1107 0 0 0 23 22 0 0 19 0 2 0 657369317 296206336 5723 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/17336/statm: 72316 5723 400 364 0 8231 0
[pid=17338] ppid=17336 vsize=19448 CPUtime=195.42 cores=5,7
/proc/17338/stat : 17338 (lingeling) T 17336 17338 17338 0 -1 4202496 18687 0 0 0 19540 2 0 0 25 0 1 0 657369318 19914752 4063 33554432000 4194304 4352236 140735518280912 18446744073709551615 4328009 0 0 16781312 17506 18446744071562691360 0 0 17 5 0 0 0
/proc/17338/statm: 4862 4063 129 39 0 3942 0
[pid=17336/tid=17353] ppid=17185 vsize=289264 CPUtime=0 cores=5,7
/proc/17336/task/17353/stat : 17353 (python) S 17185 17185 15311 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657388861 296206336 5723 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 514 18446604468773304320 0 0 -1 5 0 0 0
[pid=17354] ppid=17185 vsize=299520 CPUtime=0.13 cores=5,7
/proc/17354/stat : 17354 (python) S 17185 17185 15311 0 -1 4202560 933 0 0 0 6 7 0 0 15 0 1 0 657388862 306708480 5697 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977350 0 0 16781312 514 4085565310784995591 0 0 17 7 0 0 0
/proc/17354/statm: 74880 5697 371 364 0 10792 0
[pid=17356] ppid=17354 vsize=55004 CPUtime=51.79 cores=5,7
/proc/17356/stat : 17356 (cryptominisat-2) R 17354 17356 17356 0 -1 4202496 26744 0 0 0 5176 3 0 0 25 0 1 0 657388862 56324096 6524 33554432000 4194304 4677096 140736988409360 18446744073709551615 4454520 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/17356/statm: 13751 6524 392 118 0 8710 0
[pid=17185/tid=17219] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17219/stat : 17219 (python) S 17181 17185 15311 0 -1 4202560 33 931 0 0 0 0 0 0 18 0 4 0 657281932 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
[pid=17185/tid=17337] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17337/stat : 17337 (python) S 17181 17185 15311 0 -1 4202560 29 931 0 0 0 0 0 0 20 0 4 0 657369317 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
[pid=17185/tid=17355] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17355/stat : 17355 (python) S 17181 17185 15311 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 4 0 657388862 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1124.28
Current children cumulated vsize (KiB) 1293452

[startup+1182.3 s]
/proc/loadavg: 4.02 4.04 3.87 6/356 17371
/proc/meminfo: memFree=15447740/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=309764 CPUtime=0.75 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 11372 931 1 0 44 31 0 0 19 0 4 0 657281812 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 77441 7210 1880 364 0 13353 0
[pid=17218] ppid=17185 vsize=288880 CPUtime=1.92 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 1204 0 0 0 101 91 0 0 18 0 2 0 657281932 295813120 5658 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/17218/statm: 72220 5658 417 364 0 8138 0
[pid=17220] ppid=17218 vsize=31572 CPUtime=873.82 cores=5,7
/proc/17220/stat : 17220 (lingeling) T 17218 17220 17220 0 -1 4202496 72292 0 0 0 87373 9 0 0 25 0 1 0 657281932 32329728 7093 33554432000 4194304 4352236 140734807614176 18446744073709551615 4328194 0 0 16781312 17506 18446744071562691360 0 0 17 5 0 0 0
/proc/17220/statm: 7893 7093 129 39 0 6973 0
[pid=17218/tid=17321] ppid=17185 vsize=288880 CPUtime=0 cores=5,7
/proc/17218/task/17321/stat : 17321 (python) S 17185 17185 15311 0 -1 4202560 43 0 0 0 0 0 0 0 18 0 2 0 657340155 295813120 5658 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 514 18446604462319706176 0 0 -1 5 0 0 0
[pid=17336] ppid=17185 vsize=289264 CPUtime=0.45 cores=5,7
/proc/17336/stat : 17336 (python) S 17185 17185 15311 0 -1 4202560 1107 0 0 0 23 22 0 0 19 0 2 0 657369317 296206336 5723 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/17336/statm: 72316 5723 400 364 0 8231 0
[pid=17338] ppid=17336 vsize=19448 CPUtime=195.42 cores=5,7
/proc/17338/stat : 17338 (lingeling) T 17336 17338 17338 0 -1 4202496 18687 0 0 0 19540 2 0 0 25 0 1 0 657369318 19914752 4063 33554432000 4194304 4352236 140735518280912 18446744073709551615 4328009 0 0 16781312 17506 18446744071562691360 0 0 17 5 0 0 0
/proc/17338/statm: 4862 4063 129 39 0 3942 0
[pid=17336/tid=17353] ppid=17185 vsize=289264 CPUtime=0 cores=5,7
/proc/17336/task/17353/stat : 17353 (python) S 17185 17185 15311 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657388861 296206336 5723 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 514 18446604468773304320 0 0 -1 5 0 0 0
[pid=17354] ppid=17185 vsize=299520 CPUtime=0.27 cores=5,7
/proc/17354/stat : 17354 (python) S 17185 17185 15311 0 -1 4202560 959 0 0 0 13 14 0 0 15 0 1 0 657388862 306708480 5697 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977350 0 0 16781312 514 14055998319813951751 0 0 17 7 0 0 0
/proc/17354/statm: 74880 5697 371 364 0 10792 0
[pid=17356] ppid=17354 vsize=66412 CPUtime=111.79 cores=5,7
/proc/17356/stat : 17356 (cryptominisat-2) R 17354 17356 17356 0 -1 4202496 46042 0 0 0 11174 5 0 0 25 0 1 0 657388862 68005888 9667 33554432000 4194304 4677096 140736988409360 18446744073709551615 4320203 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/17356/statm: 16603 9667 392 118 0 11562 0
[pid=17185/tid=17219] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17219/stat : 17219 (python) S 17181 17185 15311 0 -1 4202560 33 931 0 0 0 0 0 0 18 0 4 0 657281932 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
[pid=17185/tid=17337] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17337/stat : 17337 (python) S 17181 17185 15311 0 -1 4202560 29 931 0 0 0 0 0 0 20 0 4 0 657369317 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
[pid=17185/tid=17355] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17355/stat : 17355 (python) S 17181 17185 15311 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 4 0 657388862 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1184.42
Current children cumulated vsize (KiB) 1304860

[startup+1242.3 s]
/proc/loadavg: 4.06 4.04 3.88 6/356 17373
/proc/meminfo: memFree=15416708/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=309764 CPUtime=0.75 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 11372 931 1 0 44 31 0 0 19 0 4 0 657281812 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 77441 7210 1880 364 0 13353 0
[pid=17218] ppid=17185 vsize=288880 CPUtime=1.92 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 1204 0 0 0 101 91 0 0 18 0 2 0 657281932 295813120 5658 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/17218/statm: 72220 5658 417 364 0 8138 0
[pid=17220] ppid=17218 vsize=31572 CPUtime=873.82 cores=5,7
/proc/17220/stat : 17220 (lingeling) T 17218 17220 17220 0 -1 4202496 72292 0 0 0 87373 9 0 0 25 0 1 0 657281932 32329728 7093 33554432000 4194304 4352236 140734807614176 18446744073709551615 4328194 0 0 16781312 17506 18446744071562691360 0 0 17 5 0 0 0
/proc/17220/statm: 7893 7093 129 39 0 6973 0
[pid=17218/tid=17321] ppid=17185 vsize=288880 CPUtime=0 cores=5,7
/proc/17218/task/17321/stat : 17321 (python) S 17185 17185 15311 0 -1 4202560 43 0 0 0 0 0 0 0 18 0 2 0 657340155 295813120 5658 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 514 18446604462319706176 0 0 -1 5 0 0 0
[pid=17336] ppid=17185 vsize=289264 CPUtime=0.45 cores=5,7
/proc/17336/stat : 17336 (python) S 17185 17185 15311 0 -1 4202560 1107 0 0 0 23 22 0 0 19 0 2 0 657369317 296206336 5723 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/17336/statm: 72316 5723 400 364 0 8231 0
[pid=17338] ppid=17336 vsize=19448 CPUtime=195.42 cores=5,7
/proc/17338/stat : 17338 (lingeling) T 17336 17338 17338 0 -1 4202496 18687 0 0 0 19540 2 0 0 25 0 1 0 657369318 19914752 4063 33554432000 4194304 4352236 140735518280912 18446744073709551615 4328009 0 0 16781312 17506 18446744071562691360 0 0 17 5 0 0 0
/proc/17338/statm: 4862 4063 129 39 0 3942 0
[pid=17336/tid=17353] ppid=17185 vsize=289264 CPUtime=0 cores=5,7
/proc/17336/task/17353/stat : 17353 (python) S 17185 17185 15311 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657388861 296206336 5723 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 514 18446604468773304320 0 0 -1 5 0 0 0
[pid=17354] ppid=17185 vsize=299520 CPUtime=0.43 cores=5,7
/proc/17354/stat : 17354 (python) S 17185 17185 15311 0 -1 4202560 981 0 0 0 22 21 0 0 15 0 1 0 657388862 306708480 5697 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977350 0 0 16781312 514 14055998319813951751 0 0 17 7 0 0 0
/proc/17354/statm: 74880 5697 371 364 0 10792 0
[pid=17356] ppid=17354 vsize=71620 CPUtime=171.78 cores=5,7
/proc/17356/stat : 17356 (cryptominisat-2) R 17354 17356 17356 0 -1 4202496 60535 0 0 0 17171 7 0 0 25 0 1 0 657388862 73338880 11409 33554432000 4194304 4677096 140736988409360 18446744073709551615 4318923 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/17356/statm: 17905 11409 392 118 0 12864 0
[pid=17185/tid=17219] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17219/stat : 17219 (python) S 17181 17185 15311 0 -1 4202560 33 931 0 0 0 0 0 0 18 0 4 0 657281932 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
[pid=17185/tid=17337] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17337/stat : 17337 (python) S 17181 17185 15311 0 -1 4202560 29 931 0 0 0 0 0 0 20 0 4 0 657369317 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
[pid=17185/tid=17355] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17355/stat : 17355 (python) S 17181 17185 15311 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 4 0 657388862 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1244.57
Current children cumulated vsize (KiB) 1310068



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 4.02 4.03 3.89 6/356 17374
/proc/meminfo: memFree=15373748/32951124 swapFree=52307768/67111528
[pid=17185] ppid=17181 vsize=309764 CPUtime=0.75 cores=5,7
/proc/17185/stat : 17185 (python) S 17181 17185 15311 0 -1 4202496 11372 931 1 0 44 31 0 0 19 0 4 0 657281812 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17185/statm: 77441 7210 1880 364 0 13353 0
[pid=17218] ppid=17185 vsize=288880 CPUtime=1.92 cores=5,7
/proc/17218/stat : 17218 (python) S 17185 17185 15311 0 -1 4202560 1204 0 0 0 101 91 0 0 18 0 2 0 657281932 295813120 5658 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/17218/statm: 72220 5658 417 364 0 8138 0
[pid=17220] ppid=17218 vsize=31572 CPUtime=873.82 cores=5,7
/proc/17220/stat : 17220 (lingeling) T 17218 17220 17220 0 -1 4202496 72292 0 0 0 87373 9 0 0 25 0 1 0 657281932 32329728 7093 33554432000 4194304 4352236 140734807614176 18446744073709551615 4328194 0 0 16781312 17506 18446744071562691360 0 0 17 5 0 0 0
/proc/17220/statm: 7893 7093 129 39 0 6973 0
[pid=17218/tid=17321] ppid=17185 vsize=288880 CPUtime=0 cores=5,7
/proc/17218/task/17321/stat : 17321 (python) S 17185 17185 15311 0 -1 4202560 43 0 0 0 0 0 0 0 18 0 2 0 657340155 295813120 5658 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 514 18446604462319706176 0 0 -1 5 0 0 0
[pid=17336] ppid=17185 vsize=289264 CPUtime=0.45 cores=5,7
/proc/17336/stat : 17336 (python) S 17185 17185 15311 0 -1 4202560 1107 0 0 0 23 22 0 0 19 0 2 0 657369317 296206336 5723 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/17336/statm: 72316 5723 400 364 0 8231 0
[pid=17338] ppid=17336 vsize=19448 CPUtime=195.42 cores=5,7
/proc/17338/stat : 17338 (lingeling) T 17336 17338 17338 0 -1 4202496 18687 0 0 0 19540 2 0 0 25 0 1 0 657369318 19914752 4063 33554432000 4194304 4352236 140735518280912 18446744073709551615 4328009 0 0 16781312 17506 18446744071562691360 0 0 17 5 0 0 0
/proc/17338/statm: 4862 4063 129 39 0 3942 0
[pid=17336/tid=17353] ppid=17185 vsize=289264 CPUtime=0 cores=5,7
/proc/17336/task/17353/stat : 17353 (python) S 17185 17185 15311 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657388861 296206336 5723 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 514 18446604468773304320 0 0 -1 5 0 0 0
[pid=17354] ppid=17185 vsize=299520 CPUtime=0.57 cores=5,7
/proc/17354/stat : 17354 (python) S 17185 17185 15311 0 -1 4202560 998 0 0 0 30 27 0 0 16 0 1 0 657388862 306708480 5697 33554432000 4194304 5683644 140733975272880 18446744073709551615 218370977350 0 0 16781312 514 14055998319813951751 0 0 17 7 0 0 0
/proc/17354/statm: 74880 5697 371 364 0 10792 0
[pid=17356] ppid=17354 vsize=72152 CPUtime=229.5 cores=5,7
/proc/17356/stat : 17356 (cryptominisat-2) R 17354 17356 17356 0 -1 4202496 73932 0 0 0 22939 11 0 0 25 0 1 0 657388862 73883648 10694 33554432000 4194304 4677096 140736988409360 18446744073709551615 4319536 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/17356/statm: 18038 10694 392 118 0 12997 0
[pid=17185/tid=17219] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17219/stat : 17219 (python) S 17181 17185 15311 0 -1 4202560 33 931 0 0 0 0 0 0 18 0 4 0 657281932 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
[pid=17185/tid=17337] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17337/stat : 17337 (python) S 17181 17185 15311 0 -1 4202560 29 931 0 0 0 0 0 0 20 0 4 0 657369317 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
[pid=17185/tid=17355] ppid=17181 vsize=309764 CPUtime=0 cores=5,7
/proc/17185/task/17355/stat : 17355 (python) S 17181 17185 15311 0 -1 4202560 24 931 0 0 0 0 0 0 20 0 4 0 657388862 317198336 7210 33554432000 4194304 5683644 140733975272880 18446744073709551615 218382780849 0 0 16781312 2 18446604465031351040 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1302.43
Current children cumulated vsize (KiB) 1310600

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 17185 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=452931
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=318951
# CPU time returned by wait4() is 0.771882
# while last known CPU time is 1302.43
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1300.03
CPU time (s): 1302.43
CPU user time (s): 1300.5
CPU system time (s): 1.93
CPU usage (%): 100.184
Max. virtual memory (cumulated for all children) (KiB): 1310600

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.452931
system time used= 0.318951
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 12661
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 36390
involuntary context switches= 29

runsolver used 3.89841 second user time and 9.03763 second system time

The end

Launcher Data 

Begin job on node101 at 2011-05-15 12:53:10
IDJOB=3377935
IDBENCH=83420
IDSOLVER=1858
FILE ID=node101/3377935-1305456790
RUNJOBID= node101-1305456790-17158
PBS_JOBID= 13324878
Free space on /tmp= 73756 MiB

SOLVER NAME= borg-sat 11.03.02-seq-industrial (fixed)
BENCH NAME= SAT11/application/kullmann/AES/Benchmarks/64/aes_64_1_keyfind_1.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3377935-1305456790/watcher-3377935-1305456790 -o /tmp/evaluation-result-3377935-1305456790/solver-3377935-1305456790 -C 2400 -W 1300 -M 7680  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3377935-1305456790.cnf -seed 1792784160 -budget 2400

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= 9e18e0ee8a56ea2c59fe8e682ff6c747
RANDOM SEED=1792784160

node101.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15913244 kB
Buffers:        340140 kB
Cached:        6298440 kB
SwapCached:      14848 kB
Active:       11537700 kB
Inactive:      5209636 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15913244 kB
SwapTotal:    67111528 kB
SwapFree:     52307768 kB
Dirty:          127528 kB
Writeback:           0 kB
AnonPages:    10106116 kB
Mapped:          14896 kB
Slab:           161956 kB
PageTables:      63480 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 42460916 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73632 MiB
End job on node101 at 2011-05-15 13:14:53