Trace number 3317837

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-industrial? (TO) 4800.11 1215.45

General information on the benchmark

Nameapplication/jarvisalo/HWMCC10-combination-timeframe-expansion/
hwmcc10-timeframe-expansion-k45-pdtpmspalu-tseitin.cnf
MD5SUM8dae13531986e2a9bdb6559418f57130
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark1166.09
Satisfiable
(Un)Satisfiability was proved
Number of variables89190
Number of clauses265718
Sum of the clauses size620006
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2177144
Number of clauses of size 388572
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.33/0.68	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.33/0.68	c INFO: solving HOME/instance-3317837-1304269538.cnf
0.33/0.68	c INFO: computing task features
1.10/1.45	c INFO: computed variable-clause graph statistics
1.10/1.48	c INFO: computed clause balance statistics
5.21/5.51	c INFO: computed variable balance statistics
8.10/8.40	c INFO: computed small-clause counts
8.10/8.44	c INFO: computed Horn-clause counts
17.42/17.76	c INFO: computed variable graph statistics
17.42/17.76	c INFO: collected features for HOME/instance-3317837-1304269538.cnf in 17.06 s
17.42/17.79	c INFO: running lingeling-276@0 for 97 with 4782 remaining (b = 0.40)
17.50/17.81	c INFO: running cryptominisat-2.9.0@0 for 485 with 4782 remaining (b = 0.35)
17.50/17.84	c INFO: running lingeling-276@0 for 1261 with 4782 remaining (b = 0.48)
17.50/17.86	c INFO: running cryptominisat-2.9.0@0 for 1940 with 4782 remaining (b = 0.56)
417.15/117.93	c INFO: running lingeling-276@0 for 97 with 4682 remaining (b = 0.11)
796.93/213.07	c INFO: running lingeling-276@94 for 97 with 4587 remaining (b = 0.11)
1201.10/314.21	c INFO: running lingeling-276@195 for 97 with 4486 remaining (b = 0.11)
1580.90/409.34	c INFO: running precosat-570@0 for 291 with 4391 remaining (b = 0.37)
1975.47/508.13	c INFO: running lingeling-276@0 for 97 with 4292 remaining (b = 0.21)
2375.17/608.28	c INFO: running lingeling-276@290 for 1067 with 4192 remaining (b = 0.28)
2768.93/706.80	c INFO: running cryptominisat-2.9.0@0 for 1552 with 4093 remaining (b = 0.42)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317837-1304269538/watcher-3317837-1304269538 -o /tmp/evaluation-result-3317837-1304269538/solver-3317837-1304269538 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317837-1304269538.cnf -seed 59805720 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.21 6.49 6.19 6/201 7421
/proc/meminfo: memFree=23936300/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=15184 CPUtime=0 cores=0,2,4,6
/proc/7421/stat : 7421 (python) S 7419 7421 6903 0 -1 4202496 727 0 0 0 0 0 0 0 25 0 1 0 538554270 15548416 616 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640857600 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/7421/statm: 3796 616 322 364 0 283 0

[startup+0.103827 s]
/proc/loadavg: 6.21 6.49 6.19 6/201 7421
/proc/meminfo: memFree=23936300/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=93956 CPUtime=0.05 cores=0,2,4,6
/proc/7421/stat : 7421 (python) R 7419 7421 6903 0 -1 4202496 2692 0 0 0 4 1 0 0 25 0 1 0 538554270 96210944 2406 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640857600 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/7421/statm: 23489 2406 764 364 0 1621 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 93956

[startup+0.200847 s]
/proc/loadavg: 6.21 6.49 6.19 6/201 7421
/proc/meminfo: memFree=23936300/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=124068 CPUtime=0.1 cores=0,2,4,6
/proc/7421/stat : 7421 (python) S 7419 7421 6903 0 -1 4202496 4242 0 1 0 8 2 0 0 25 0 1 0 538554270 127045632 3850 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640857600 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/7421/statm: 31017 3850 991 364 0 2828 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 124068

[startup+0.300797 s]
/proc/loadavg: 6.21 6.49 6.19 6/201 7421
/proc/meminfo: memFree=23936300/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=135748 CPUtime=0.17 cores=0,2,4,6
/proc/7421/stat : 7421 (python) S 7419 7421 6903 0 -1 4202496 5962 934 1 0 13 4 0 0 17 0 1 0 538554270 139005952 4150 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640855557 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/7421/statm: 33937 4150 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135748

[startup+0.70076 s]
/proc/loadavg: 6.21 6.49 6.19 6/201 7421
/proc/meminfo: memFree=23936300/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=283408 CPUtime=0.4 cores=0,2,4,6
/proc/7421/stat : 7421 (python) R 7419 7421 6903 0 -1 4202496 10489 934 1 0 29 11 0 0 16 0 1 0 538554270 290209792 8171 33554432000 4194304 5683644 140734685416816 18446744073709551615 4962267 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/7421/statm: 70852 8172 1761 364 0 6776 0
Current children cumulated CPU time (s) 0.4
Current children cumulated vsize (KiB) 283408

[startup+1.50064 s]
/proc/loadavg: 6.21 6.49 6.19 6/202 7426
/proc/meminfo: memFree=23880988/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=331772 CPUtime=1.2 cores=0,2,4,6
/proc/7421/stat : 7421 (python) R 7419 7421 6903 0 -1 4202496 28563 934 1 0 107 13 0 0 17 0 1 0 538554270 339734528 20328 33554432000 4194304 5683644 140734685416816 18446744073709551615 47392937386398 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/7421/statm: 82943 20328 1826 364 0 18867 0
Current children cumulated CPU time (s) 1.2
Current children cumulated vsize (KiB) 331772

[startup+3.10032 s]
/proc/loadavg: 6.21 6.49 6.19 6/202 7426
/proc/meminfo: memFree=23856436/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=331772 CPUtime=2.8 cores=0,2,4,6
/proc/7421/stat : 7421 (python) R 7419 7421 6903 0 -1 4202496 28563 934 1 0 267 13 0 0 19 0 1 0 538554270 339734528 20328 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640526819 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/7421/statm: 82943 20328 1826 364 0 18867 0
Current children cumulated CPU time (s) 2.8
Current children cumulated vsize (KiB) 331772

[startup+6.30074 s]
/proc/loadavg: 6.12 6.47 6.18 6/202 7426
/proc/meminfo: memFree=23913468/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=331772 CPUtime=6 cores=0,2,4,6
/proc/7421/stat : 7421 (python) R 7419 7421 6903 0 -1 4202496 28564 934 1 0 587 13 0 0 25 0 1 0 538554270 339734528 20329 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640545995 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/7421/statm: 82943 20329 1827 364 0 18867 0
Current children cumulated CPU time (s) 6
Current children cumulated vsize (KiB) 331772

[startup+12.7006 s]
/proc/loadavg: 6.03 6.44 6.17 6/202 7426
/proc/meminfo: memFree=23904980/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=331772 CPUtime=12.41 cores=0,2,4,6
/proc/7421/stat : 7421 (python) R 7419 7421 6903 0 -1 4202496 28570 934 1 0 1227 14 0 0 25 0 1 0 538554270 339734528 20335 33554432000 4194304 5683644 140734685416816 18446744073709551615 47392934623372 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/7421/statm: 82943 20335 1833 364 0 18867 0
Current children cumulated CPU time (s) 12.41
Current children cumulated vsize (KiB) 331772

[startup+25.5012 s]
/proc/loadavg: 6.26 6.47 6.19 9/214 7439
/proc/meminfo: memFree=23546896/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=368736 CPUtime=17.57 cores=0,2,4,6
/proc/7421/stat : 7421 (python) S 7419 7421 6903 0 -1 4202496 31203 934 1 0 1742 15 0 0 22 0 5 0 538554270 377585664 19381 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/7421/statm: 92184 19381 1881 364 0 28093 0
[pid=7428] ppid=7421 vsize=327724 CPUtime=0 cores=0,2,4,6
/proc/7428/stat : 7428 (python) S 7421 7421 6903 0 -1 4202560 980 0 0 0 0 0 0 0 18 0 1 0 538556050 335589376 17865 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879631 0 0 16781312 514 4374925984890061060 0 0 17 2 0 0 0
/proc/7428/statm: 81931 17865 381 364 0 17849 0
[pid=7430] ppid=7428 vsize=31348 CPUtime=7.69 cores=0,2,4,6
/proc/7430/stat : 7430 (lingeling) R 7428 7430 7430 0 -1 4202496 14320 0 0 0 768 1 0 0 25 0 1 0 538556050 32100352 7007 33554432000 4194304 4352236 140737198958640 18446744073709551615 4207619 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/7430/statm: 7837 7007 129 39 0 6917 0
[pid=7431] ppid=7421 vsize=337980 CPUtime=0.01 cores=0,2,4,6
/proc/7431/stat : 7431 (python) S 7421 7421 6903 0 -1 4202560 903 0 0 0 1 0 0 0 15 0 1 0 538556052 346091520 17858 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 3099338565042012420 0 0 17 0 0 0 0
/proc/7431/statm: 84495 17858 370 364 0 20410 0
[pid=7433] ppid=7431 vsize=163672 CPUtime=7.64 cores=0,2,4,6
/proc/7433/stat : 7433 (cryptominisat-2) R 7431 7433 7433 0 -1 4202496 42120 0 0 0 759 5 0 0 25 0 1 0 538556052 167600128 33166 33554432000 4194304 4677096 140735619729408 18446744073709551615 4322145 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/7433/statm: 40918 33166 393 118 0 35877 0
[pid=7434] ppid=7421 vsize=348236 CPUtime=0 cores=0,2,4,6
/proc/7434/stat : 7434 (python) S 7421 7421 6903 0 -1 4202560 926 0 0 0 0 0 0 0 18 0 1 0 538556055 356593664 17862 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 4375354519546986756 0 0 17 6 0 0 0
/proc/7434/statm: 87059 17862 370 364 0 22971 0
[pid=7436] ppid=7434 vsize=31324 CPUtime=7.59 cores=0,2,4,6
/proc/7436/stat : 7436 (lingeling) R 7434 7436 7436 0 -1 4202496 14931 0 0 0 757 2 0 0 25 0 1 0 538556055 32075776 7001 33554432000 4194304 4352236 140733390808192 18446744073709551615 4332999 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/7436/statm: 7831 7001 129 39 0 6911 0
[pid=7437] ppid=7421 vsize=358492 CPUtime=0.01 cores=0,2,4,6
/proc/7437/stat : 7437 (python) S 7421 7421 6903 0 -1 4202560 940 0 0 0 0 1 0 0 15 0 1 0 538556057 367095808 17865 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 4449055058834718980 0 0 17 0 0 0 0
/proc/7437/statm: 89623 17865 369 364 0 25532 0
[pid=7439] ppid=7437 vsize=167024 CPUtime=7.61 cores=0,2,4,6
/proc/7439/stat : 7439 (cryptominisat-2) R 7437 7439 7439 0 -1 4202496 42075 0 0 0 756 5 0 0 25 0 1 0 538556058 171032576 34069 33554432000 4194304 4677096 140735713100496 18446744073709551615 4319761 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/7439/statm: 41756 34069 393 118 0 36715 0
[pid=7421/tid=7429] ppid=7419 vsize=368736 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7429/stat : 7429 (python) S 7419 7421 6903 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 5 0 538556050 377585664 19381 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 2 0 0 0
[pid=7421/tid=7432] ppid=7419 vsize=368736 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7432/stat : 7432 (python) S 7419 7421 6903 0 -1 4202560 21 934 0 0 0 0 0 0 25 0 5 0 538556052 377585664 19381 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7435] ppid=7419 vsize=368736 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7435/stat : 7435 (python) S 7419 7421 6903 0 -1 4202560 23 934 0 0 0 0 0 0 20 0 5 0 538556055 377585664 19381 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7438] ppid=7419 vsize=368736 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7438/stat : 7438 (python) S 7419 7421 6903 0 -1 4202560 25 934 0 0 0 0 0 0 22 0 5 0 538556057 377585664 19381 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 48.12
Current children cumulated vsize (KiB) 2134536

[startup+51.1016 s]
/proc/loadavg: 6.85 6.59 6.23 9/214 7439
/proc/meminfo: memFree=23439504/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=368736 CPUtime=17.57 cores=0,2,4,6
/proc/7421/stat : 7421 (python) S 7419 7421 6903 0 -1 4202496 31203 934 1 0 1742 15 0 0 22 0 5 0 538554270 377585664 19381 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/7421/statm: 92184 19381 1881 364 0 28093 0
[pid=7428] ppid=7421 vsize=327724 CPUtime=0.03 cores=0,2,4,6
/proc/7428/stat : 7428 (python) S 7421 7421 6903 0 -1 4202560 994 0 0 0 1 2 0 0 18 0 1 0 538556050 335589376 17865 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879631 0 0 16781312 514 9511320938029744388 0 0 17 2 0 0 0
/proc/7428/statm: 81931 17865 381 364 0 17849 0
[pid=7430] ppid=7428 vsize=31348 CPUtime=33.3 cores=0,2,4,6
/proc/7430/stat : 7430 (lingeling) R 7428 7430 7430 0 -1 4202496 14320 0 0 0 3328 2 0 0 25 0 1 0 538556050 32100352 7007 33554432000 4194304 4352236 140737198958640 18446744073709551615 4238671 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/7430/statm: 7837 7007 129 39 0 6917 0
[pid=7431] ppid=7421 vsize=337980 CPUtime=0.06 cores=0,2,4,6
/proc/7431/stat : 7431 (python) S 7421 7421 6903 0 -1 4202560 936 0 0 0 4 2 0 0 15 0 1 0 538556052 346091520 17858 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 3099338565042012420 0 0 17 0 0 0 0
/proc/7431/statm: 84495 17858 370 364 0 20410 0
[pid=7433] ppid=7431 vsize=205272 CPUtime=33.13 cores=0,2,4,6
/proc/7433/stat : 7433 (cryptominisat-2) R 7431 7433 7433 0 -1 4202496 62861 0 0 0 3305 8 0 0 25 0 1 0 538556052 210198528 43670 33554432000 4194304 4677096 140735619729408 18446744073709551615 4320291 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/7433/statm: 51318 43670 394 118 0 46277 0
[pid=7434] ppid=7421 vsize=348236 CPUtime=0.03 cores=0,2,4,6
/proc/7434/stat : 7434 (python) S 7421 7421 6903 0 -1 4202560 940 0 0 0 2 1 0 0 18 0 1 0 538556055 356593664 17862 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 9511320938029744388 0 0 17 6 0 0 0
/proc/7434/statm: 87059 17862 370 364 0 22971 0
[pid=7436] ppid=7434 vsize=31324 CPUtime=33.01 cores=0,2,4,6
/proc/7436/stat : 7436 (lingeling) R 7434 7436 7436 0 -1 4202496 14931 0 0 0 3299 2 0 0 25 0 1 0 538556055 32075776 7001 33554432000 4194304 4352236 140733390808192 18446744073709551615 4270879 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/7436/statm: 7831 7001 129 39 0 6911 0
[pid=7437] ppid=7421 vsize=358492 CPUtime=0.05 cores=0,2,4,6
/proc/7437/stat : 7437 (python) S 7421 7421 6903 0 -1 4202560 959 0 0 0 2 3 0 0 15 0 1 0 538556057 367095808 17866 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 4449055058834718980 0 0 17 0 0 0 0
/proc/7437/statm: 89623 17866 370 364 0 25532 0
[pid=7439] ppid=7437 vsize=205880 CPUtime=33.18 cores=0,2,4,6
/proc/7439/stat : 7439 (cryptominisat-2) R 7437 7439 7439 0 -1 4202496 63058 0 0 0 3310 8 0 0 25 0 1 0 538556058 210821120 43196 33554432000 4194304 4677096 140735713100496 18446744073709551615 4338016 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/7439/statm: 51470 43196 394 118 0 46429 0
[pid=7421/tid=7429] ppid=7419 vsize=368736 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7429/stat : 7429 (python) S 7419 7421 6903 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 5 0 538556050 377585664 19381 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 2 0 0 0
[pid=7421/tid=7432] ppid=7419 vsize=368736 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7432/stat : 7432 (python) S 7419 7421 6903 0 -1 4202560 21 934 0 0 0 0 0 0 25 0 5 0 538556052 377585664 19381 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7435] ppid=7419 vsize=368736 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7435/stat : 7435 (python) S 7419 7421 6903 0 -1 4202560 23 934 0 0 0 0 0 0 20 0 5 0 538556055 377585664 19381 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7438] ppid=7419 vsize=368736 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7438/stat : 7438 (python) S 7419 7421 6903 0 -1 4202560 25 934 0 0 0 0 0 0 22 0 5 0 538556057 377585664 19381 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 150.36
Current children cumulated vsize (KiB) 2214992

[startup+102.301 s]
/proc/loadavg: 7.55 6.82 6.33 9/218 7445
/proc/meminfo: memFree=23309460/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=368736 CPUtime=17.57 cores=0,2,4,6
/proc/7421/stat : 7421 (python) S 7419 7421 6903 0 -1 4202496 31203 934 1 0 1742 15 0 0 22 0 5 0 538554270 377585664 19381 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/7421/statm: 92184 19381 1881 364 0 28093 0
[pid=7428] ppid=7421 vsize=327724 CPUtime=0.1 cores=0,2,4,6
/proc/7428/stat : 7428 (python) S 7421 7421 6903 0 -1 4202560 1016 0 0 0 5 5 0 0 18 0 1 0 538556050 335589376 17865 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879631 0 0 16781312 514 9511320938029744388 0 0 17 2 0 0 0
/proc/7428/statm: 81931 17865 381 364 0 17849 0
[pid=7430] ppid=7428 vsize=36012 CPUtime=84.51 cores=0,2,4,6
/proc/7430/stat : 7430 (lingeling) R 7428 7430 7430 0 -1 4202496 23678 0 0 0 8448 3 0 0 25 0 1 0 538556050 36876288 8173 33554432000 4194304 4352236 140737198958640 18446744073709551615 4276815 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/7430/statm: 9003 8173 129 39 0 8083 0
[pid=7431] ppid=7421 vsize=337980 CPUtime=0.14 cores=0,2,4,6
/proc/7431/stat : 7431 (python) S 7421 7421 6903 0 -1 4202560 966 0 0 0 8 6 0 0 15 0 1 0 538556052 346091520 17859 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 3099338565042012420 0 0 17 0 0 0 0
/proc/7431/statm: 84495 17859 371 364 0 20410 0
[pid=7433] ppid=7431 vsize=212948 CPUtime=84.18 cores=0,2,4,6
/proc/7433/stat : 7433 (cryptominisat-2) R 7431 7433 7433 0 -1 4202496 85974 0 0 0 8404 14 0 0 25 0 1 0 538556052 218058752 45221 33554432000 4194304 4677096 140735619729408 18446744073709551615 4317843 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/7433/statm: 53237 45221 394 118 0 48196 0

################
# More data... #
################

/proc/7557/statm: 11314 10484 129 39 0 10394 0
[pid=7555/tid=7564] ppid=7421 vsize=379004 CPUtime=0 cores=0,2,4,6
/proc/7555/task/7564/stat : 7564 (python) S 7421 7421 6903 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538615097 388100096 17904 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 514 18446604453904140224 0 0 -1 0 0 0 0
[pid=7421/tid=7429] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7429/stat : 7429 (python) S 7419 7421 6903 0 -1 4202560 37 934 0 0 0 0 0 0 18 0 7 0 538556050 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 2 0 0 0
[pid=7421/tid=7432] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7432/stat : 7432 (python) S 7419 7421 6903 0 -1 4202560 27 934 0 0 0 0 0 0 18 0 7 0 538556052 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7435] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7435/stat : 7435 (python) S 7419 7421 6903 0 -1 4202560 23 934 0 0 0 0 0 0 20 0 7 0 538556055 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7438] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7438/stat : 7438 (python) S 7419 7421 6903 0 -1 4202560 25 934 0 0 0 0 0 0 22 0 7 0 538556057 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7505] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7505/stat : 7505 (python) S 7419 7421 6903 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 7 0 538595205 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7556] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7556/stat : 7556 (python) S 7419 7421 6903 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 7 0 538605084 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4667.89
Current children cumulated vsize (KiB) 3445156



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1215.4 s]
/proc/loadavg: 8.18 8.10 7.55 9/227 7586
/proc/meminfo: memFree=25255928/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=389248 CPUtime=17.73 cores=0,2,4,6
/proc/7421/stat : 7421 (python) S 7419 7421 6903 0 -1 4202496 33452 934 1 0 1757 16 0 0 18 0 7 0 538554270 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/7421/statm: 97312 19394 1886 364 0 33215 0
[pid=7428] ppid=7421 vsize=337968 CPUtime=1.33 cores=0,2,4,6
/proc/7428/stat : 7428 (python) S 7421 7421 6903 0 -1 4202560 1216 0 0 0 73 60 0 0 18 0 2 0 538556050 346079232 17904 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/7428/statm: 84492 17904 415 364 0 20410 0
[pid=7430] ppid=7428 vsize=73340 CPUtime=995.94 cores=0,2,4,6
/proc/7430/stat : 7430 (lingeling) R 7428 7430 7430 0 -1 4202496 82931 0 0 0 99578 16 0 0 25 0 1 0 538556050 75100160 17505 33554432000 4194304 4352236 140737198958640 18446744073709551615 4328259 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/7430/statm: 18335 17505 129 39 0 17415 0
[pid=7428/tid=7446] ppid=7421 vsize=337968 CPUtime=0 cores=0,2,4,6
/proc/7428/task/7446/stat : 7446 (python) S 7421 7421 6903 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 538566062 346079232 17904 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 514 18446604453931310080 0 0 -1 2 0 0 0
[pid=7431] ppid=7421 vsize=337980 CPUtime=1.5 cores=0,2,4,6
/proc/7431/stat : 7431 (python) S 7421 7421 6903 0 -1 4202560 1197 0 0 0 81 69 0 0 15 0 2 0 538556052 346091520 17889 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/7431/statm: 84495 17889 400 364 0 20410 0
[pid=7433] ppid=7431 vsize=293288 CPUtime=998.53 cores=0,2,4,6
/proc/7433/stat : 7433 (cryptominisat-2) R 7431 7433 7433 0 -1 4202496 295787 0 0 0 99782 71 0 0 25 0 1 0 538556052 300326912 64843 33554432000 4194304 4677096 140735619729408 18446744073709551615 4321064 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/7433/statm: 73322 64843 407 118 0 68281 0
[pid=7431/tid=7554] ppid=7421 vsize=337980 CPUtime=0 cores=0,2,4,6
/proc/7431/task/7554/stat : 7554 (python) S 7421 7421 6903 0 -1 4202560 49 0 0 0 0 0 0 0 16 0 2 0 538605082 346091520 17889 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 514 18446604453931410752 0 0 -1 0 0 0 0
[pid=7434] ppid=7421 vsize=348236 CPUtime=1.61 cores=0,2,4,6
/proc/7434/stat : 7434 (python) S 7421 7421 6903 0 -1 4202560 1007 0 0 0 79 82 0 0 18 0 1 0 538556055 356593664 17862 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 9511320938029744388 0 0 17 6 0 0 0
/proc/7434/statm: 87059 17862 370 364 0 22971 0
[pid=7436] ppid=7434 vsize=104444 CPUtime=1188.29 cores=0,2,4,6
/proc/7436/stat : 7436 (lingeling) R 7434 7436 7436 0 -1 4202496 105404 0 0 0 118814 15 0 0 25 0 1 0 538556055 106950656 25281 33554432000 4194304 4352236 140733390808192 18446744073709551615 4328194 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/7436/statm: 26111 25281 129 39 0 25191 0
[pid=7437] ppid=7421 vsize=358492 CPUtime=1.78 cores=0,2,4,6
/proc/7437/stat : 7437 (python) S 7421 7421 6903 0 -1 4202560 1062 0 0 0 99 79 0 0 15 0 1 0 538556057 367095808 17867 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 4449055058834718980 0 0 17 0 0 0 0
/proc/7437/statm: 89623 17867 371 364 0 25532 0
[pid=7439] ppid=7437 vsize=403396 CPUtime=1196.2 cores=0,2,4,6
/proc/7439/stat : 7439 (cryptominisat-2) R 7437 7439 7439 0 -1 4202496 315173 0 0 0 119549 71 0 0 25 0 1 0 538556058 413077504 69144 33554432000 4194304 4677096 140735713100496 18446744073709551615 4551501 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/7439/statm: 100849 69144 395 118 0 95808 0
[pid=7504] ppid=7421 vsize=368748 CPUtime=0.39 cores=0,2,4,6
/proc/7504/stat : 7504 (python) S 7421 7421 6903 0 -1 4202560 1171 0 0 0 21 18 0 0 19 0 2 0 538595205 377597952 17900 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652686283 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/7504/statm: 92187 17900 399 364 0 28093 0
[pid=7506] ppid=7504 vsize=91188 CPUtime=296.99 cores=0,2,4,6
/proc/7506/stat : 7506 (precosat) T 7504 7506 7506 0 -1 4202496 52770 0 0 0 29690 9 0 0 25 0 1 0 538595205 93376512 21406 33554432000 4194304 4890832 140737412726128 18446744073709551615 4213152 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/7506/statm: 22797 21406 86 171 0 22624 0
[pid=7504/tid=7568] ppid=7421 vsize=368748 CPUtime=0 cores=0,2,4,6
/proc/7504/task/7568/stat : 7568 (python) S 7421 7421 6903 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538624949 377597952 17900 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 514 18446604453938110848 0 0 -1 0 0 0 0
[pid=7555] ppid=7421 vsize=379004 CPUtime=0.13 cores=0,2,4,6
/proc/7555/stat : 7555 (python) S 7421 7421 6903 0 -1 4202560 1140 0 0 0 6 7 0 0 19 0 2 0 538605084 388100096 17904 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652686283 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/7555/statm: 94751 17904 399 364 0 30654 0
[pid=7557] ppid=7555 vsize=45256 CPUtime=99.69 cores=0,2,4,6
/proc/7557/stat : 7557 (lingeling) T 7555 7557 7557 0 -1 4202496 26466 0 0 0 9966 3 0 0 25 0 1 0 538605085 46342144 10484 33554432000 4194304 4352236 140735095144448 18446744073709551615 4328013 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/7557/statm: 11314 10484 129 39 0 10394 0
[pid=7555/tid=7564] ppid=7421 vsize=379004 CPUtime=0 cores=0,2,4,6
/proc/7555/task/7564/stat : 7564 (python) S 7421 7421 6903 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538615097 388100096 17904 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 514 18446604453904140224 0 0 -1 0 0 0 0
[pid=7421/tid=7429] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7429/stat : 7429 (python) S 7419 7421 6903 0 -1 4202560 37 934 0 0 0 0 0 0 18 0 7 0 538556050 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 2 0 0 0
[pid=7421/tid=7432] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7432/stat : 7432 (python) S 7419 7421 6903 0 -1 4202560 27 934 0 0 0 0 0 0 18 0 7 0 538556052 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7435] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7435/stat : 7435 (python) S 7419 7421 6903 0 -1 4202560 23 934 0 0 0 0 0 0 20 0 7 0 538556055 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7438] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7438/stat : 7438 (python) S 7419 7421 6903 0 -1 4202560 25 934 0 0 0 0 0 0 22 0 7 0 538556057 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7505] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7505/stat : 7505 (python) S 7419 7421 6903 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 7 0 538595205 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7556] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7556/stat : 7556 (python) S 7419 7421 6903 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 7 0 538605084 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.11
Current children cumulated vsize (KiB) 3530588

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1215.4 s]
/proc/loadavg: 8.18 8.10 7.55 9/227 7586
/proc/meminfo: memFree=25255928/32951124 swapFree=67111364/67111528
[pid=7421] ppid=7419 vsize=389248 CPUtime=17.73 cores=0,2,4,6
/proc/7421/stat : 7421 (python) S 7419 7421 6903 0 -1 4202496 33452 934 1 0 1757 16 0 0 18 0 7 0 538554270 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/7421/statm: 97312 19394 1886 364 0 33215 0
[pid=7428] ppid=7421 vsize=337968 CPUtime=1.33 cores=0,2,4,6
/proc/7428/stat : 7428 (python) S 7421 7421 6903 0 -1 4202560 1216 0 0 0 73 60 0 0 18 0 2 0 538556050 346079232 17904 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/7428/statm: 84492 17904 415 364 0 20410 0
[pid=7430] ppid=7428 vsize=73340 CPUtime=995.94 cores=0,2,4,6
/proc/7430/stat : 7430 (lingeling) R 7428 7430 7430 0 -1 4202496 82931 0 0 0 99578 16 0 0 25 0 1 0 538556050 75100160 17505 33554432000 4194304 4352236 140737198958640 18446744073709551615 4328259 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/7430/statm: 18335 17505 129 39 0 17415 0
[pid=7428/tid=7446] ppid=7421 vsize=337968 CPUtime=0 cores=0,2,4,6
/proc/7428/task/7446/stat : 7446 (python) S 7421 7421 6903 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 538566062 346079232 17904 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 514 18446604453931310080 0 0 -1 2 0 0 0
[pid=7431] ppid=7421 vsize=337980 CPUtime=1.5 cores=0,2,4,6
/proc/7431/stat : 7431 (python) S 7421 7421 6903 0 -1 4202560 1197 0 0 0 81 69 0 0 15 0 2 0 538556052 346091520 17889 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/7431/statm: 84495 17889 400 364 0 20410 0
[pid=7433] ppid=7431 vsize=293288 CPUtime=998.53 cores=0,2,4,6
/proc/7433/stat : 7433 (cryptominisat-2) R 7431 7433 7433 0 -1 4202496 295787 0 0 0 99782 71 0 0 25 0 1 0 538556052 300326912 64843 33554432000 4194304 4677096 140735619729408 18446744073709551615 4321064 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/7433/statm: 73322 64843 407 118 0 68281 0
[pid=7431/tid=7554] ppid=7421 vsize=337980 CPUtime=0 cores=0,2,4,6
/proc/7431/task/7554/stat : 7554 (python) S 7421 7421 6903 0 -1 4202560 49 0 0 0 0 0 0 0 16 0 2 0 538605082 346091520 17889 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 514 18446604453931410752 0 0 -1 0 0 0 0
[pid=7434] ppid=7421 vsize=348236 CPUtime=1.61 cores=0,2,4,6
/proc/7434/stat : 7434 (python) S 7421 7421 6903 0 -1 4202560 1007 0 0 0 79 82 0 0 18 0 1 0 538556055 356593664 17862 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 9511320938029744388 0 0 17 6 0 0 0
/proc/7434/statm: 87059 17862 370 364 0 22971 0
[pid=7436] ppid=7434 vsize=104444 CPUtime=1188.29 cores=0,2,4,6
/proc/7436/stat : 7436 (lingeling) R 7434 7436 7436 0 -1 4202496 105404 0 0 0 118814 15 0 0 25 0 1 0 538556055 106950656 25281 33554432000 4194304 4352236 140733390808192 18446744073709551615 4328194 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/7436/statm: 26111 25281 129 39 0 25191 0
[pid=7437] ppid=7421 vsize=358492 CPUtime=1.78 cores=0,2,4,6
/proc/7437/stat : 7437 (python) S 7421 7421 6903 0 -1 4202560 1062 0 0 0 99 79 0 0 15 0 1 0 538556057 367095808 17867 33554432000 4194304 5683644 140734685416816 18446744073709551615 253640879686 0 0 16781312 514 4449055058834718980 0 0 17 0 0 0 0
/proc/7437/statm: 89623 17867 371 364 0 25532 0
[pid=7439] ppid=7437 vsize=403396 CPUtime=1196.2 cores=0,2,4,6
/proc/7439/stat : 7439 (cryptominisat-2) R 7437 7439 7439 0 -1 4202496 315173 0 0 0 119549 71 0 0 25 0 1 0 538556058 413077504 69144 33554432000 4194304 4677096 140735713100496 18446744073709551615 4551501 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/7439/statm: 100849 69144 395 118 0 95808 0
[pid=7504] ppid=7421 vsize=368748 CPUtime=0.39 cores=0,2,4,6
/proc/7504/stat : 7504 (python) S 7421 7421 6903 0 -1 4202560 1171 0 0 0 21 18 0 0 19 0 2 0 538595205 377597952 17900 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652686283 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/7504/statm: 92187 17900 399 364 0 28093 0
[pid=7506] ppid=7504 vsize=91188 CPUtime=296.99 cores=0,2,4,6
/proc/7506/stat : 7506 (precosat) T 7504 7506 7506 0 -1 4202496 52770 0 0 0 29690 9 0 0 25 0 1 0 538595205 93376512 21406 33554432000 4194304 4890832 140737412726128 18446744073709551615 4213152 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/7506/statm: 22797 21406 86 171 0 22624 0
[pid=7504/tid=7568] ppid=7421 vsize=368748 CPUtime=0 cores=0,2,4,6
/proc/7504/task/7568/stat : 7568 (python) S 7421 7421 6903 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538624949 377597952 17900 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 514 18446604453938110848 0 0 -1 0 0 0 0
[pid=7555] ppid=7421 vsize=379004 CPUtime=0.13 cores=0,2,4,6
/proc/7555/stat : 7555 (python) S 7421 7421 6903 0 -1 4202560 1140 0 0 0 6 7 0 0 19 0 2 0 538605084 388100096 17904 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652686283 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/7555/statm: 94751 17904 399 364 0 30654 0
[pid=7557] ppid=7555 vsize=45256 CPUtime=99.69 cores=0,2,4,6
/proc/7557/stat : 7557 (lingeling) T 7555 7557 7557 0 -1 4202496 26466 0 0 0 9966 3 0 0 25 0 1 0 538605085 46342144 10484 33554432000 4194304 4352236 140735095144448 18446744073709551615 4328013 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/7557/statm: 11314 10484 129 39 0 10394 0
[pid=7555/tid=7564] ppid=7421 vsize=379004 CPUtime=0 cores=0,2,4,6
/proc/7555/task/7564/stat : 7564 (python) S 7421 7421 6903 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538615097 388100096 17904 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 514 18446604453904140224 0 0 -1 0 0 0 0
[pid=7421/tid=7429] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7429/stat : 7429 (python) S 7419 7421 6903 0 -1 4202560 37 934 0 0 0 0 0 0 18 0 7 0 538556050 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 2 0 0 0
[pid=7421/tid=7432] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7432/stat : 7432 (python) S 7419 7421 6903 0 -1 4202560 27 934 0 0 0 0 0 0 18 0 7 0 538556052 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7435] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7435/stat : 7435 (python) S 7419 7421 6903 0 -1 4202560 23 934 0 0 0 0 0 0 20 0 7 0 538556055 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7438] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7438/stat : 7438 (python) S 7419 7421 6903 0 -1 4202560 25 934 0 0 0 0 0 0 22 0 7 0 538556057 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7505] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7505/stat : 7505 (python) S 7419 7421 6903 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 7 0 538595205 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
[pid=7421/tid=7556] ppid=7419 vsize=389248 CPUtime=0 cores=0,2,4,6
/proc/7421/task/7556/stat : 7556 (python) S 7419 7421 6903 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 7 0 538605084 398589952 19394 33554432000 4194304 5683644 140734685416816 18446744073709551615 253652683185 0 0 16781312 2 18446604453922757632 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.11
Current children cumulated vsize (KiB) 3530588

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 7421 and gives
#  childrusage.ru_utime.tv_sec=17
#  childrusage.ru_utime.tv_usec=582327
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=165974
# CPU time returned by wait4() is 17.7483
# while last known CPU time is 4800.11
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1215.45
CPU time (s): 4800.11
CPU user time (s): 4794.95
CPU system time (s): 5.16
CPU usage (%): 394.925
Max. virtual memory (cumulated for all children) (KiB): 3555336

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 17.5823
system time used= 0.165974
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 34388
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2888
involuntary context switches= 52

runsolver used 2.29565 second user time and 5.40418 second system time

The end

Launcher Data 

Begin job on node106 at 2011-05-01 19:05:39
IDJOB=3317837
IDBENCH=82467
IDSOLVER=1706
FILE ID=node106/3317837-1304269538
RUNJOBID= node106-1304267295-6921
PBS_JOBID= 13173972
Free space on /tmp= 72928 MiB

SOLVER NAME= borg-sat 11.03.02-industrial
BENCH NAME= SAT11/application/jarvisalo/HWMCC10-combination-timeframe-expansion/hwmcc10-timeframe-expansion-k45-pdtpmspalu-tseitin.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317837-1304269538/watcher-3317837-1304269538 -o /tmp/evaluation-result-3317837-1304269538/solver-3317837-1304269538 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317837-1304269538.cnf -seed 59805720 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8dae13531986e2a9bdb6559418f57130
RANDOM SEED=59805720

node106.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23936340 kB
Buffers:        298872 kB
Cached:        2588580 kB
SwapCached:         88 kB
Active:        7054252 kB
Inactive:      1721712 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23936340 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:           36136 kB
Writeback:          32 kB
AnonPages:     5888368 kB
Mapped:          22880 kB
Slab:           154484 kB
PageTables:      25532 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  9095864 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72912 MiB
End job on node106 at 2011-05-01 19:25:56