Trace number 3317718

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-industrial? (TO) 4800.27 1298.18

General information on the benchmark

Nameindustrial/anbulagan/
hard-unsat/dated-10-17-u.cnf
MD5SUM780d946576ec5746b4ea9832bb01dbc5
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark1033.89
Satisfiable
(Un)Satisfiability was proved
Number of variables229544
Number of clauses1070757
Sum of the clauses size2471122
Maximum clause length10
Minimum clause length1
Number of clauses of size 119156
Number of clauses of size 2831957
Number of clauses of size 3176604
Number of clauses of size 44780
Number of clauses of size 519120
Number of clauses of size over 519140

Solver Data 

0.56/19.53	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.56/19.53	c INFO: solving HOME/instance-3317718-1304267283.cnf
0.56/19.53	c INFO: computing task features
4.23/23.24	c INFO: computed variable-clause graph statistics
4.34/23.37	c INFO: computed clause balance statistics
21.44/40.47	c INFO: computed variable balance statistics
34.54/53.56	c INFO: computed small-clause counts
34.84/53.87	c INFO: computed Horn-clause counts
101.84/120.85	c INFO: computed variable graph statistics
101.84/120.85	c INFO: collected features for HOME/instance-3317718-1304267283.cnf in 101.19 s
101.84/120.88	c INFO: running cryptominisat-2.9.0@0 for 485 with 4698 remaining (b = 0.35)
101.94/120.91	c INFO: running SApperloT@0 for 97 with 4698 remaining (b = 0.09)
101.94/120.93	c INFO: running lingeling-276@0 for 97 with 4698 remaining (b = 0.12)
101.94/120.96	c INFO: running lingeling-276@0 for 1067 with 4698 remaining (b = 0.28)
497.91/220.14	c INFO: running cryptominisat-2.9.0@0 for 1649 with 4599 remaining (b = 0.39)
501.48/221.09	c INFO: running lingeling-276@0 for 1261 with 4598 remaining (b = 0.26)
2060.01/611.31	c INFO: running cryptominisat-2.9.0@0 for 97 with 4208 remaining (b = 0.06)
2469.66/713.95	c INFO: running precosat-570@0 for 388 with 4105 remaining (b = 0.11)
4044.59/1108.46	c INFO: running lingeling-276@0 for 97 with 3711 remaining (b = 0.04)
4406.69/1199.35	c INFO: running cryptominisat-2.9.0@0 for 1746 with 3620 remaining (b = 0.33)
4443.71/1208.63	c INFO: running lingeling-276@1072 for 194 with 3610 remaining (b = 0.04)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317718-1304267283/watcher-3317718-1304267283 -o /tmp/evaluation-result-3317718-1304267283/solver-3317718-1304267283 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317718-1304267283.cnf -seed 638206346 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.48 0.10 0.03 3/171 9950
/proc/meminfo: memFree=30526184/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=11800 CPUtime=0 cores=0,2,4,6
/proc/9950/stat : 9950 (runsolver) R 9948 9950 9915 0 -1 4202560 43 0 0 0 0 0 0 0 20 0 1 0 538328710 12083200 51 33554432000 4194304 5434984 140735353489472 18446744073709551615 4881607 0 2147483391 4096 24578 0 0 0 17 6 0 0 0
/proc/9950/statm: 2950 51 31 303 0 2644 0

[startup+0.0566721 s]
/proc/loadavg: 0.48 0.10 0.03 3/171 9950
/proc/meminfo: memFree=30526184/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=14384 CPUtime=0 cores=0,2,4,6
/proc/9950/stat : 9950 (python) D 9948 9950 9915 0 -1 4194304 233 0 5 0 0 0 0 0 20 0 1 0 538328710 14729216 169 33554432000 4194304 5683644 140733427811920 18446744073709551615 4689677 0 0 4096 0 18446744071562234166 0 0 17 0 0 0 2
/proc/9950/statm: 3596 169 135 364 0 83 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 14384

[startup+0.100662 s]
/proc/loadavg: 0.48 0.10 0.03 3/171 9950
/proc/meminfo: memFree=30526184/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=14820 CPUtime=0 cores=0,2,4,6
/proc/9950/stat : 9950 (python) S 9948 9950 9915 0 -1 4202496 507 0 6 0 0 0 0 0 20 0 1 0 538328710 15175680 420 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664804197 0 2147483391 4096 0 18446744071699109054 0 0 17 0 0 0 2
/proc/9950/statm: 3705 420 259 364 0 192 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 14820

[startup+0.300593 s]
/proc/loadavg: 0.48 0.10 0.03 3/171 9950
/proc/meminfo: memFree=30526184/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=15756 CPUtime=0 cores=0,2,4,6
/proc/9950/stat : 9950 (python) S 9948 9950 9915 0 -1 4202496 834 0 6 0 0 0 0 0 16 0 1 0 538328710 16134144 723 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664804197 0 2147483391 16781312 2 18446744071699855507 0 0 17 0 0 0 5
/proc/9950/statm: 3939 723 334 364 0 426 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15756

[startup+0.700578 s]
/proc/loadavg: 0.48 0.10 0.03 3/171 9950
/proc/meminfo: memFree=30526184/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=16816 CPUtime=0.04 cores=0,2,4,6
/proc/9950/stat : 9950 (python) D 9948 9950 9915 0 -1 4202496 1223 0 6 0 2 2 0 0 16 0 1 0 538328710 17219584 1086 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664806400 0 0 16781312 2 0 0 0 17 0 0 0 6
/proc/9950/statm: 4204 1086 417 364 0 691 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 16816

[startup+1.50036 s]
/proc/loadavg: 0.48 0.10 0.03 1/176 9955
/proc/meminfo: memFree=30514768/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=36840 CPUtime=0.07 cores=0,2,4,6
/proc/9950/stat : 9950 (python) D 9948 9950 9915 0 -1 4202496 1610 0 13 0 4 3 0 0 15 0 1 0 538328710 37724160 1452 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664806400 0 0 16781312 2 0 0 0 17 0 0 0 9
/proc/9950/statm: 9210 1452 452 364 0 1027 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 36840

[startup+3.10107 s]
/proc/loadavg: 0.48 0.10 0.03 1/176 9955
/proc/meminfo: memFree=30508840/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=94188 CPUtime=0.12 cores=0,2,4,6
/proc/9950/stat : 9950 (python) D 9948 9950 9915 0 -1 4202496 2744 0 24 0 6 6 0 0 15 0 1 0 538328710 96448512 2475 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664808192 0 0 16781312 2 18446744071562234166 0 0 17 0 0 0 19
/proc/9950/statm: 23547 2475 764 364 0 1679 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 94188
heavy processes:

[startup+6.30056 s]
/proc/loadavg: 0.68 0.15 0.05 1/176 9955
/proc/meminfo: memFree=30497812/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=107432 CPUtime=0.14 cores=0,2,4,6
/proc/9950/stat : 9950 (python) D 9948 9950 9915 0 -1 4202496 3370 0 33 0 7 7 0 0 16 0 1 0 538328710 110010368 3058 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664806400 0 0 16781312 2 0 0 0 17 0 0 0 121
/proc/9950/statm: 26858 3058 871 364 0 2089 0
Current children cumulated CPU time (s) 0.14
Current children cumulated vsize (KiB) 107432
heavy processes:

[startup+12.7004 s]
/proc/loadavg: 0.87 0.19 0.06 1/180 9959
/proc/meminfo: memFree=30484136/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=124240 CPUtime=0.19 cores=0,2,4,6
/proc/9950/stat : 9950 (python) S 9948 9950 9915 0 -1 4202496 4287 0 41 0 10 9 0 0 17 0 1 0 538328710 127221760 3891 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664808192 0 0 16781312 2 18446744071562362840 0 0 17 0 0 0 153
/proc/9950/statm: 31060 3891 997 364 0 2871 0
[pid=9957] ppid=9950 vsize=8700 CPUtime=0 cores=0,2,4,6
/proc/9957/stat : 9957 (sh) S 9950 9950 9915 0 -1 4202496 307 0 0 0 0 0 0 0 18 0 1 0 538329656 8908800 245 33554432000 4194304 4922060 140736341023808 18446744073709551615 274664627333 0 65536 16781316 65538 18446744071562232103 0 0 17 2 0 0 0
/proc/9957/statm: 2175 245 206 178 0 68 0
[pid=9958] ppid=9957 vsize=980 CPUtime=0 cores=0,2,4,6
/proc/9958/stat : 9958 (ldconfig) D 9957 9950 9915 0 -1 4202496 141 0 8 0 0 0 0 0 18 0 1 0 538329656 1003520 84 33554432000 4194304 4893030 140733647766064 18446744073709551615 4304218 0 0 16781312 0 18446744071562234166 0 0 17 4 0 0 284
/proc/9958/statm: 245 84 73 171 0 59 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 133920
heavy processes:

[startup+25.5002 s]
/proc/loadavg: 1.50 0.36 0.12 3/176 9963
/proc/meminfo: memFree=30166120/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=476560 CPUtime=6.54 cores=0,2,4,6
/proc/9950/stat : 9950 (python) R 9948 9950 9915 0 -1 4202496 93255 923 109 9 617 37 0 0 25 0 1 0 538328710 487997440 56524 33554432000 4194304 5683644 140733427811920 18446744073709551615 47852312763789 0 0 16781312 2 0 0 0 17 0 0 0 614
/proc/9950/statm: 119140 56524 1826 364 0 55064 0
Current children cumulated CPU time (s) 6.54
Current children cumulated vsize (KiB) 476560
heavy processes:

[startup+51.1008 s]
/proc/loadavg: 1.67 0.49 0.17 3/176 10008
/proc/meminfo: memFree=30165208/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=476560 CPUtime=32.14 cores=0,2,4,6
/proc/9950/stat : 9950 (python) R 9948 9950 9915 0 -1 4202496 93256 923 109 9 3177 37 0 0 25 0 1 0 538328710 487997440 56525 33554432000 4194304 5683644 140733427811920 18446744073709551615 4528400 0 0 16781312 2 0 0 0 17 0 0 0 614
/proc/9950/statm: 119140 56525 1827 364 0 55064 0
Current children cumulated CPU time (s) 32.14
Current children cumulated vsize (KiB) 476560
heavy processes:

[startup+102.306 s]
/proc/loadavg: 1.98 0.77 0.28 3/176 10009
/proc/meminfo: memFree=30031500/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=484924 CPUtime=83.34 cores=0,2,4,6
/proc/9950/stat : 9950 (python) R 9948 9950 9915 0 -1 4202496 95353 923 109 9 8297 37 0 0 25 0 1 0 538328710 496562176 58622 33554432000 4194304 5683644 140733427811920 18446744073709551615 47852312709248 0 0 16781312 2 0 0 0 17 0 0 0 614
/proc/9950/statm: 121231 58622 1833 364 0 57155 0
Current children cumulated CPU time (s) 83.34
Current children cumulated vsize (KiB) 484924

[startup+162.301 s]
/proc/loadavg: 5.04 1.78 0.65 9/200 10035
/proc/meminfo: memFree=28662568/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=509704 CPUtime=102.01 cores=0,2,4,6
/proc/9950/stat : 9950 (python) S 9948 9950 9915 0 -1 4202496 98112 923 109 9 10160 41 0 0 21 0 5 0 538328710 521936896 54622 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676635083 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 614
/proc/9950/statm: 127426 54622 1881 364 0 63335 0
[pid=10023] ppid=9950 vsize=468692 CPUtime=0.06 cores=0,2,4,6
/proc/10023/stat : 10023 (python) S 9950 9950 9915 0 -1 4202560 1027 0 0 0 3 3 0 0 15 0 1 0 538340799 479940608 53107 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828431 0 0 16781312 514 3448569846302736644 0 0 17 4 0 0 0
/proc/10023/statm: 117173 53107 382 364 0 53091 0
[pid=10025] ppid=10023 vsize=676264 CPUtime=41.28 cores=0,2,4,6
/proc/10025/stat : 10025 (cryptominisat-2) R 10023 10025 10025 0 -1 4202496 179377 0 0 0 4106 22 0 0 25 0 1 0 538340801 692494336 141292 33554432000 4194304 4677096 140735352076608 18446744073709551615 4320006 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/10025/statm: 169066 141292 393 118 0 164025 0
[pid=10026] ppid=9950 vsize=478948 CPUtime=0.05 cores=0,2,4,6
/proc/10026/stat : 10026 (python) S 9950 9950 9915 0 -1 4202560 931 0 0 0 3 2 0 0 18 0 1 0 538340801 490442752 53099 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828486 0 0 16781312 514 8723560439146840323 0 0 17 0 0 0 0
/proc/10026/statm: 119737 53099 370 364 0 55652 0
[pid=10028] ppid=10026 vsize=75648 CPUtime=41.31 cores=0,2,4,6
/proc/10028/stat : 10028 (SApperloT-base) R 10026 10028 10028 0 -1 4202496 18699 0 0 0 4125 6 0 0 25 0 1 0 538340802 77463552 17211 33554432000 134512640 135674940 4292557424 18446744073709551615 134564025 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/10028/statm: 18912 17211 99 284 0 18626 0
[pid=10029] ppid=9950 vsize=489204 CPUtime=0.05 cores=0,2,4,6
/proc/10029/stat : 10029 (python) S 9950 9950 9915 0 -1 4202560 947 0 0 0 3 2 0 0 18 0 1 0 538340804 500944896 53103 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828486 0 0 16781312 514 8723560439146840321 0 0 17 2 0 0 0
/proc/10029/statm: 122301 53103 370 364 0 58213 0
[pid=10033] ppid=10029 vsize=27248 CPUtime=41.2 cores=0,2,4,6
/proc/10033/stat : 10033 (lingeling) R 10029 10033 10033 0 -1 4202496 23094 0 0 0 4115 5 0 0 25 0 1 0 538340808 27901952 5937 33554432000 4194304 4352236 140734059490032 18446744073709551615 4241377 0 0 16781312 17506 0 0 0 17 2 0 0 0
/proc/10033/statm: 6812 5937 129 39 0 5892 0
[pid=10031] ppid=9950 vsize=499460 CPUtime=0.05 cores=0,2,4,6
/proc/10031/stat : 10031 (python) S 9950 9950 9915 0 -1 4202560 954 0 0 0 3 2 0 0 18 0 1 0 538340807 511447040 53107 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828486 0 0 16781312 514 8723560439146840323 0 0 17 4 0 0 0
/proc/10031/statm: 124865 53107 370 364 0 60774 0
[pid=10034] ppid=10031 vsize=27024 CPUtime=41.03 cores=0,2,4,6
/proc/10034/stat : 10034 (lingeling) R 10031 10034 10034 0 -1 4202496 23067 0 0 0 4099 4 0 0 25 0 1 0 538340809 27672576 5925 33554432000 4194304 4352236 140733910020736 18446744073709551615 4328026 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/10034/statm: 6756 5925 129 39 0 5836 0
[pid=9950/tid=10024] ppid=9948 vsize=509704 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10024/stat : 10024 (python) S 9948 9950 9915 0 -1 4202560 24 923 0 9 0 0 0 0 25 0 5 0 538340799 521936896 54622 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 6 0 0 0
[pid=9950/tid=10027] ppid=9948 vsize=509704 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10027/stat : 10027 (python) S 9948 9950 9915 0 -1 4202560 22 923 0 9 0 0 0 0 25 0 5 0 538340802 521936896 54622 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 6 0 0 0
[pid=9950/tid=10030] ppid=9948 vsize=509704 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10030/stat : 10030 (python) S 9948 9950 9915 0 -1 4202560 40 923 0 9 0 0 0 0 20 0 5 0 538340805 521936896 54622 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
[pid=9950/tid=10032] ppid=9948 vsize=509704 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10032/stat : 10032 (python) S 9948 9950 9915 0 -1 4202560 19 923 0 9 0 0 0 0 22 0 5 0 538340808 521936896 54622 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 267.04
Current children cumulated vsize (KiB) 3252192

[startup+222.301 s]
/proc/loadavg: 6.91 2.91 1.11 9/210 10047
/proc/meminfo: memFree=28038876/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=519960 CPUtime=102.06 cores=0,2,4,6
/proc/9950/stat : 9950 (python) S 9948 9950 9915 0 -1 4202496 99453 923 109 9 10164 42 0 0 18 0 6 0 538328710 532439040 54629 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676635083 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 614
/proc/9950/statm: 129990 54629 1884 364 0 65896 0
[pid=10023] ppid=9950 vsize=468692 CPUtime=0.16 cores=0,2,4,6
/proc/10023/stat : 10023 (python) S 9950 9950 9915 0 -1 4202560 1065 0 0 0 9 7 0 0 15 0 1 0 538340799 479940608 53108 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828431 0 0 16781312 514 3448569846302736644 0 0 17 4 0 0 0
/proc/10023/statm: 117173 53108 383 364 0 53091 0
[pid=10025] ppid=10023 vsize=724692 CPUtime=101.1 cores=0,2,4,6
/proc/10025/stat : 10025 (cryptominisat-2) R 10023 10025 10025 0 -1 4202496 207035 0 0 0 10083 27 0 0 25 0 1 0 538340801 742084608 153047 33554432000 4194304 4677096 140735352076608 18446744073709551615 4576264 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/10025/statm: 181173 153047 393 118 0 176132 0
[pid=10026] ppid=9950 vsize=478948 CPUtime=0.13 cores=0,2,4,6
/proc/10026/stat : 10026 (python) S 9950 9950 9915 0 -1 4202560 1111 0 0 0 7 6 0 0 19 0 2 0 538340801 490442752 53129 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676635083 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/10026/statm: 119737 53129 399 364 0 55652 0
[pid=10028] ppid=10026 vsize=83924 CPUtime=99.99 cores=0,2,4,6
/proc/10028/stat : 10028 (SApperloT-base) T 10026 10028 10028 0 -1 4202496 20599 0 0 0 9992 7 0 0 25 0 1 0 538340802 85938176 19111 33554432000 134512640 135674940 4292557424 18446744073709551615 134564070 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/10028/statm: 20981 19111 100 284 0 20695 0
[pid=10026/tid=10047] ppid=9950 vsize=478948 CPUtime=0 cores=0,2,4,6
/proc/10026/task/10047/stat : 10047 (python) S 9950 9950 9915 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538350817 490442752 53129 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604441826050176 0 0 -1 0 0 0 0
[pid=10029] ppid=9950 vsize=489204 CPUtime=0.14 cores=0,2,4,6

################
# More data... #
################

[pid=10029] ppid=9950 vsize=489204 CPUtime=1.58 cores=0,2,4,6
/proc/10029/stat : 10029 (python) S 9950 9950 9915 0 -1 4202560 1160 0 0 0 88 70 0 0 18 0 2 0 538340804 500944896 53133 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828486 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/10029/statm: 122301 53133 399 364 0 58213 0
[pid=10033] ppid=10029 vsize=66248 CPUtime=1172.81 cores=0,2,4,6
/proc/10033/stat : 10033 (lingeling) R 10029 10033 10033 0 -1 4202496 67746 0 0 0 117253 28 0 0 25 0 1 0 538340808 67837952 15732 33554432000 4194304 4352236 140734059490032 18446744073709551615 4328026 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/10033/statm: 16562 15732 129 39 0 15642 0
[pid=10029/tid=10043] ppid=9950 vsize=489204 CPUtime=0 cores=0,2,4,6
/proc/10029/task/10043/stat : 10043 (python) S 9950 9950 9915 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 538350723 500944896 53133 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604453720904832 0 0 -1 2 0 0 0
[pid=10031] ppid=9950 vsize=499460 CPUtime=1.43 cores=0,2,4,6
/proc/10031/stat : 10031 (python) S 9950 9950 9915 0 -1 4202560 1161 0 0 0 77 66 0 0 19 0 2 0 538340807 511447040 53137 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676635083 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/10031/statm: 124865 53137 399 364 0 60774 0
[pid=10034] ppid=10031 vsize=48012 CPUtime=1072.27 cores=0,2,4,6
/proc/10034/stat : 10034 (lingeling) T 10031 10034 10034 0 -1 4202496 64195 0 0 0 107217 10 0 0 25 0 1 0 538340809 49164288 11172 33554432000 4194304 4352236 140733910020736 18446744073709551615 4328194 0 0 16781312 17506 18446744071562691360 0 0 17 6 0 0 0
/proc/10034/statm: 12003 11172 129 39 0 11083 0
[pid=10031/tid=10100] ppid=9950 vsize=499460 CPUtime=0 cores=0,2,4,6
/proc/10031/task/10100/stat : 10100 (python) S 9950 9950 9915 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 538448643 511447040 53137 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604453880480896 0 0 -1 4 0 0 0
[pid=10044] ppid=9950 vsize=509716 CPUtime=1.63 cores=0,2,4,6
/proc/10044/stat : 10044 (python) S 9950 9950 9915 0 -1 4202560 1096 0 0 0 92 71 0 0 15 0 1 0 538350725 521949184 53112 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828486 0 0 16781312 514 4251305693555949826 0 0 17 0 0 0 0
/proc/10044/statm: 127429 53112 371 364 0 63335 0
[pid=10046] ppid=10044 vsize=838120 CPUtime=1075.23 cores=0,2,4,6
/proc/10046/stat : 10046 (cryptominisat-2) R 10044 10046 10046 0 -1 4202496 240041 0 0 0 107478 45 0 0 25 0 1 0 538350726 858234880 167940 33554432000 4194304 4677096 140736334188064 18446744073709551615 4355205 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/10046/statm: 209530 167940 393 118 0 204489 0
[pid=10063] ppid=9950 vsize=519972 CPUtime=0.18 cores=0,2,4,6
/proc/10063/stat : 10063 (python) S 9950 9950 9915 0 -1 4202560 1177 0 0 0 10 8 0 0 16 0 2 0 538389842 532451328 53145 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676635083 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/10063/statm: 129993 53145 399 364 0 65896 0
[pid=10065] ppid=10063 vsize=810912 CPUtime=102.26 cores=0,2,4,6
/proc/10065/stat : 10065 (cryptominisat-2) T 10063 10065 10065 0 -1 4202496 339436 0 0 0 10180 46 0 0 25 0 1 0 538389842 830373888 175342 33554432000 4194304 4677096 140736346778368 18446744073709551615 274664808320 0 0 16781312 2 18446744071562691360 0 0 17 4 0 0 0
/proc/10065/statm: 202728 175342 393 118 0 197687 0
[pid=10063/tid=10072] ppid=9950 vsize=519972 CPUtime=0 cores=0,2,4,6
/proc/10063/task/10072/stat : 10072 (python) S 9950 9950 9915 0 -1 4202560 52 0 0 0 0 0 0 0 16 0 2 0 538400103 532451328 53145 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604440278878080 0 0 -1 4 0 0 0
[pid=10073] ppid=9950 vsize=530228 CPUtime=0.53 cores=0,2,4,6
/proc/10073/stat : 10073 (python) S 9950 9950 9915 0 -1 4202560 1163 0 0 0 27 26 0 0 19 0 2 0 538400106 542953472 53149 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676635083 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/10073/statm: 132557 53149 399 364 0 68457 0
[pid=10075] ppid=10073 vsize=247376 CPUtime=391.79 cores=0,2,4,6
/proc/10075/stat : 10075 (precosat) T 10073 10075 10075 0 -1 4202496 64983 0 0 0 39169 10 0 0 25 0 1 0 538400107 253313024 61131 33554432000 4194304 4890832 140736873504496 18446744073709551615 4238686 0 0 16781312 17442 18446744071562691360 0 0 17 6 0 0 0
/proc/10075/statm: 61844 61131 86 171 0 61671 0
[pid=10073/tid=10093] ppid=9950 vsize=530228 CPUtime=0 cores=0,2,4,6
/proc/10073/task/10093/stat : 10093 (python) S 9950 9950 9915 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 538439554 542953472 53149 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604453713606848 0 0 -1 4 0 0 0
[pid=10094] ppid=9950 vsize=540484 CPUtime=0.27 cores=0,2,4,6
/proc/10094/stat : 10094 (python) S 9950 9950 9915 0 -1 4202560 1194 0 0 0 13 14 0 0 18 0 2 0 538439556 553455616 53159 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828486 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/10094/statm: 135121 53159 405 364 0 71018 0
[pid=10096] ppid=10094 vsize=34160 CPUtime=188.92 cores=0,2,4,6
/proc/10096/stat : 10096 (lingeling) R 10094 10096 10096 0 -1 4202496 31013 0 0 0 18887 5 0 0 25 0 1 0 538439557 34979840 7710 33554432000 4194304 4352236 140734619118576 18446744073709551615 4328194 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/10096/statm: 8540 7710 129 39 0 7620 0
[pid=10094/tid=10101] ppid=9950 vsize=540484 CPUtime=0 cores=0,2,4,6
/proc/10094/task/10101/stat : 10101 (python) S 9950 9950 9915 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538449571 553455616 53159 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604444788302976 0 0 -1 4 0 0 0
[pid=9950/tid=10024] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10024/stat : 10024 (python) S 9948 9950 9915 0 -1 4202560 29 923 0 9 0 0 0 0 18 0 9 0 538340799 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 6 0 0 0
[pid=9950/tid=10027] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10027/stat : 10027 (python) S 9948 9950 9915 0 -1 4202560 22 923 0 9 0 0 0 0 25 0 9 0 538340802 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 6 0 0 0
[pid=9950/tid=10030] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10030/stat : 10030 (python) S 9948 9950 9915 0 -1 4202560 45 923 0 9 0 0 0 0 18 0 9 0 538340805 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
[pid=9950/tid=10032] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10032/stat : 10032 (python) S 9948 9950 9915 0 -1 4202560 19 923 0 9 0 0 0 0 22 0 9 0 538340808 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
[pid=9950/tid=10045] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10045/stat : 10045 (python) S 9948 9950 9915 0 -1 4202560 32 923 0 9 0 0 0 0 20 0 9 0 538350726 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
[pid=9950/tid=10064] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10064/stat : 10064 (python) S 9948 9950 9915 0 -1 4202560 27 923 0 9 0 0 0 0 20 0 9 0 538389842 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
[pid=9950/tid=10074] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10074/stat : 10074 (python) S 9948 9950 9915 0 -1 4202560 26 923 0 9 0 0 0 0 20 0 9 0 538400107 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
[pid=9950/tid=10095] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10095/stat : 10095 (python) S 9948 9950 9915 0 -1 4202560 30 923 0 9 0 0 0 0 18 0 9 0 538439557 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 7496064

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1298.1 s]
/proc/loadavg: 8.16 8.00 5.91 9/241 10106
/proc/meminfo: memFree=25042360/32951132 swapFree=67111356/67111528
[pid=9950] ppid=9948 vsize=550728 CPUtime=102.2 cores=0,2,4,6
/proc/9950/stat : 9950 (python) S 9948 9950 9915 0 -1 4202496 101997 923 109 9 10176 44 0 0 18 0 9 0 538328710 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676635083 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 614
/proc/9950/statm: 137682 54647 1890 364 0 73579 0
[pid=10023] ppid=9950 vsize=478936 CPUtime=0.9 cores=0,2,4,6
/proc/10023/stat : 10023 (python) S 9950 9950 9915 0 -1 4202560 1260 0 0 0 53 37 0 0 15 0 2 0 538340799 490430464 53147 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828486 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/10023/statm: 119734 53147 417 364 0 55652 0
[pid=10025] ppid=10023 vsize=769636 CPUtime=588.15 cores=0,2,4,6
/proc/10025/stat : 10025 (cryptominisat-2) R 10023 10025 10025 0 -1 4202496 228568 0 0 0 58771 44 0 0 25 0 1 0 538340801 788107264 165555 33554432000 4194304 4677096 140735352076608 18446744073709551615 4320006 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/10025/statm: 192409 165555 393 118 0 187368 0
[pid=10023/tid=10062] ppid=9950 vsize=478936 CPUtime=0 cores=0,2,4,6
/proc/10023/task/10062/stat : 10062 (python) S 9950 9950 9915 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 538389839 490430464 53147 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604453734852800 0 0 -1 4 0 0 0
[pid=10026] ppid=9950 vsize=478948 CPUtime=0.13 cores=0,2,4,6
/proc/10026/stat : 10026 (python) S 9950 9950 9915 0 -1 4202560 1111 0 0 0 7 6 0 0 19 0 2 0 538340801 490442752 53129 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676635083 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/10026/statm: 119737 53129 399 364 0 55652 0
[pid=10028] ppid=10026 vsize=83924 CPUtime=99.99 cores=0,2,4,6
/proc/10028/stat : 10028 (SApperloT-base) T 10026 10028 10028 0 -1 4202496 20599 0 0 0 9992 7 0 0 25 0 1 0 538340802 85938176 19111 33554432000 134512640 135674940 4292557424 18446744073709551615 134564070 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/10028/statm: 20981 19111 100 284 0 20695 0
[pid=10026/tid=10047] ppid=9950 vsize=478948 CPUtime=0 cores=0,2,4,6
/proc/10026/task/10047/stat : 10047 (python) S 9950 9950 9915 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538350817 490442752 53129 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604441826050176 0 0 -1 0 0 0 0
[pid=10029] ppid=9950 vsize=489204 CPUtime=1.58 cores=0,2,4,6
/proc/10029/stat : 10029 (python) S 9950 9950 9915 0 -1 4202560 1160 0 0 0 88 70 0 0 18 0 2 0 538340804 500944896 53133 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828486 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/10029/statm: 122301 53133 399 364 0 58213 0
[pid=10033] ppid=10029 vsize=66248 CPUtime=1172.81 cores=0,2,4,6
/proc/10033/stat : 10033 (lingeling) R 10029 10033 10033 0 -1 4202496 67746 0 0 0 117253 28 0 0 25 0 1 0 538340808 67837952 15732 33554432000 4194304 4352236 140734059490032 18446744073709551615 4328026 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/10033/statm: 16562 15732 129 39 0 15642 0
[pid=10029/tid=10043] ppid=9950 vsize=489204 CPUtime=0 cores=0,2,4,6
/proc/10029/task/10043/stat : 10043 (python) S 9950 9950 9915 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 538350723 500944896 53133 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604453720904832 0 0 -1 2 0 0 0
[pid=10031] ppid=9950 vsize=499460 CPUtime=1.43 cores=0,2,4,6
/proc/10031/stat : 10031 (python) S 9950 9950 9915 0 -1 4202560 1161 0 0 0 77 66 0 0 19 0 2 0 538340807 511447040 53137 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676635083 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/10031/statm: 124865 53137 399 364 0 60774 0
[pid=10034] ppid=10031 vsize=48012 CPUtime=1072.27 cores=0,2,4,6
/proc/10034/stat : 10034 (lingeling) T 10031 10034 10034 0 -1 4202496 64195 0 0 0 107217 10 0 0 25 0 1 0 538340809 49164288 11172 33554432000 4194304 4352236 140733910020736 18446744073709551615 4328194 0 0 16781312 17506 18446744071562691360 0 0 17 6 0 0 0
/proc/10034/statm: 12003 11172 129 39 0 11083 0
[pid=10031/tid=10100] ppid=9950 vsize=499460 CPUtime=0 cores=0,2,4,6
/proc/10031/task/10100/stat : 10100 (python) S 9950 9950 9915 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 538448643 511447040 53137 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604453880480896 0 0 -1 4 0 0 0
[pid=10044] ppid=9950 vsize=509716 CPUtime=1.63 cores=0,2,4,6
/proc/10044/stat : 10044 (python) S 9950 9950 9915 0 -1 4202560 1096 0 0 0 92 71 0 0 15 0 1 0 538350725 521949184 53112 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828486 0 0 16781312 514 4251305693555949826 0 0 17 0 0 0 0
/proc/10044/statm: 127429 53112 371 364 0 63335 0
[pid=10046] ppid=10044 vsize=838120 CPUtime=1075.23 cores=0,2,4,6
/proc/10046/stat : 10046 (cryptominisat-2) R 10044 10046 10046 0 -1 4202496 240041 0 0 0 107478 45 0 0 25 0 1 0 538350726 858234880 167940 33554432000 4194304 4677096 140736334188064 18446744073709551615 4355205 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/10046/statm: 209530 167940 393 118 0 204489 0
[pid=10063] ppid=9950 vsize=519972 CPUtime=0.18 cores=0,2,4,6
/proc/10063/stat : 10063 (python) S 9950 9950 9915 0 -1 4202560 1177 0 0 0 10 8 0 0 16 0 2 0 538389842 532451328 53145 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676635083 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/10063/statm: 129993 53145 399 364 0 65896 0
[pid=10065] ppid=10063 vsize=810912 CPUtime=102.26 cores=0,2,4,6
/proc/10065/stat : 10065 (cryptominisat-2) T 10063 10065 10065 0 -1 4202496 339436 0 0 0 10180 46 0 0 25 0 1 0 538389842 830373888 175342 33554432000 4194304 4677096 140736346778368 18446744073709551615 274664808320 0 0 16781312 2 18446744071562691360 0 0 17 4 0 0 0
/proc/10065/statm: 202728 175342 393 118 0 197687 0
[pid=10063/tid=10072] ppid=9950 vsize=519972 CPUtime=0 cores=0,2,4,6
/proc/10063/task/10072/stat : 10072 (python) S 9950 9950 9915 0 -1 4202560 52 0 0 0 0 0 0 0 16 0 2 0 538400103 532451328 53145 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604440278878080 0 0 -1 4 0 0 0
[pid=10073] ppid=9950 vsize=530228 CPUtime=0.53 cores=0,2,4,6
/proc/10073/stat : 10073 (python) S 9950 9950 9915 0 -1 4202560 1163 0 0 0 27 26 0 0 19 0 2 0 538400106 542953472 53149 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676635083 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/10073/statm: 132557 53149 399 364 0 68457 0
[pid=10075] ppid=10073 vsize=247376 CPUtime=391.79 cores=0,2,4,6
/proc/10075/stat : 10075 (precosat) T 10073 10075 10075 0 -1 4202496 64983 0 0 0 39169 10 0 0 25 0 1 0 538400107 253313024 61131 33554432000 4194304 4890832 140736873504496 18446744073709551615 4238686 0 0 16781312 17442 18446744071562691360 0 0 17 6 0 0 0
/proc/10075/statm: 61844 61131 86 171 0 61671 0
[pid=10073/tid=10093] ppid=9950 vsize=530228 CPUtime=0 cores=0,2,4,6
/proc/10073/task/10093/stat : 10093 (python) S 9950 9950 9915 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 538439554 542953472 53149 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604453713606848 0 0 -1 4 0 0 0
[pid=10094] ppid=9950 vsize=540484 CPUtime=0.27 cores=0,2,4,6
/proc/10094/stat : 10094 (python) S 9950 9950 9915 0 -1 4202560 1194 0 0 0 13 14 0 0 18 0 2 0 538439556 553455616 53159 33554432000 4194304 5683644 140733427811920 18446744073709551615 274664828486 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/10094/statm: 135121 53159 405 364 0 71018 0
[pid=10096] ppid=10094 vsize=34160 CPUtime=188.92 cores=0,2,4,6
/proc/10096/stat : 10096 (lingeling) R 10094 10096 10096 0 -1 4202496 31013 0 0 0 18887 5 0 0 25 0 1 0 538439557 34979840 7710 33554432000 4194304 4352236 140734619118576 18446744073709551615 4328194 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/10096/statm: 8540 7710 129 39 0 7620 0
[pid=10094/tid=10101] ppid=9950 vsize=540484 CPUtime=0 cores=0,2,4,6
/proc/10094/task/10101/stat : 10101 (python) S 9950 9950 9915 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538449571 553455616 53159 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 514 18446604444788302976 0 0 -1 4 0 0 0
[pid=9950/tid=10024] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10024/stat : 10024 (python) S 9948 9950 9915 0 -1 4202560 29 923 0 9 0 0 0 0 18 0 9 0 538340799 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 6 0 0 0
[pid=9950/tid=10027] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10027/stat : 10027 (python) S 9948 9950 9915 0 -1 4202560 22 923 0 9 0 0 0 0 25 0 9 0 538340802 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 6 0 0 0
[pid=9950/tid=10030] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10030/stat : 10030 (python) S 9948 9950 9915 0 -1 4202560 45 923 0 9 0 0 0 0 18 0 9 0 538340805 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
[pid=9950/tid=10032] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10032/stat : 10032 (python) S 9948 9950 9915 0 -1 4202560 19 923 0 9 0 0 0 0 22 0 9 0 538340808 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
[pid=9950/tid=10045] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10045/stat : 10045 (python) S 9948 9950 9915 0 -1 4202560 32 923 0 9 0 0 0 0 20 0 9 0 538350726 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
[pid=9950/tid=10064] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10064/stat : 10064 (python) S 9948 9950 9915 0 -1 4202560 27 923 0 9 0 0 0 0 20 0 9 0 538389842 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
[pid=9950/tid=10074] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10074/stat : 10074 (python) S 9948 9950 9915 0 -1 4202560 26 923 0 9 0 0 0 0 20 0 9 0 538400107 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
[pid=9950/tid=10095] ppid=9948 vsize=550728 CPUtime=0 cores=0,2,4,6
/proc/9950/task/10095/stat : 10095 (python) S 9948 9950 9915 0 -1 4202560 30 923 0 9 0 0 0 0 18 0 9 0 538439557 563945472 54647 33554432000 4194304 5683644 140733427811920 18446744073709551615 274676631985 0 0 16781312 2 18446604453720903936 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 7496064

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 9950 and gives
#  childrusage.ru_utime.tv_sec=101
#  childrusage.ru_utime.tv_usec=768528
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=459930
# CPU time returned by wait4() is 102.228
# while last known CPU time is 4800.27
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1298.18
CPU time (s): 4800.27
CPU user time (s): 4794.9
CPU system time (s): 5.37
CPU usage (%): 369.768
Max. virtual memory (cumulated for all children) (KiB): 7557188

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 101.769
system time used= 0.45993
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 102926
page faults= 118
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 15138
involuntary context switches= 242

runsolver used 2.84757 second user time and 6.85296 second system time

The end

Launcher Data 

Begin job on node131 at 2011-05-01 18:28:04
IDJOB=3317718
IDBENCH=20384
IDSOLVER=1706
FILE ID=node131/3317718-1304267283
RUNJOBID= node131-1304267281-9933
PBS_JOBID= 13173948
Free space on /tmp= 72224 MiB

SOLVER NAME= borg-sat 11.03.02-industrial
BENCH NAME= SAT07/industrial/anbulagan/hard-unsat/dated-10-17-u.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317718-1304267283/watcher-3317718-1304267283 -o /tmp/evaluation-result-3317718-1304267283/solver-3317718-1304267283 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317718-1304267283.cnf -seed 638206346 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 780d946576ec5746b4ea9832bb01dbc5
RANDOM SEED=638206346

node131.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      30526348 kB
Buffers:        277696 kB
Cached:        1765852 kB
SwapCached:        168 kB
Active:         416888 kB
Inactive:      1692492 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      30526348 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:           89344 kB
Writeback:           0 kB
AnonPages:       65964 kB
Mapped:          14764 kB
Slab:           253624 kB
PageTables:       4656 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS:   188696 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72144 MiB
End job on node131 at 2011-05-01 18:49:51