Trace number 3317867

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-industrial? (TO) 4800.29 1206.55

General information on the benchmark

NameSAT03/industrial/li/li-exam/
li-exam-61.shuffled-as.sat03-366.cnf
MD5SUM3be662940f394fa876137a905247d755
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables28147
Number of clauses108436
Sum of the clauses size373110
Maximum clause length166
Minimum clause length1
Number of clauses of size 114
Number of clauses of size 218786
Number of clauses of size 351588
Number of clauses of size 437440
Number of clauses of size 53
Number of clauses of size over 5605

Solver Data 

0.34/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.34/0.67	c INFO: solving HOME/instance-3317867-1304270918.cnf
0.34/0.67	c INFO: computing task features
0.73/1.10	c INFO: computed variable-clause graph statistics
0.84/1.12	c INFO: computed clause balance statistics
2.94/3.27	c INFO: computed variable balance statistics
4.43/4.72	c INFO: computed small-clause counts
4.43/4.73	c INFO: computed Horn-clause counts
5.53/5.81	c INFO: computed variable graph statistics
5.53/5.81	c INFO: collected features for HOME/instance-3317867-1304270918.cnf in 5.13 s
5.53/5.84	c INFO: running lingeling-276@0 for 97 with 4794 remaining (b = 0.30)
5.53/5.86	c INFO: running cryptominisat-2.9.0@0 for 485 with 4794 remaining (b = 0.32)
5.53/5.88	c INFO: running lingeling-276@0 for 1261 with 4794 remaining (b = 0.48)
5.62/5.91	c INFO: running lingeling-276@0 for 1261 with 4794 remaining (b = 0.41)
405.41/106.00	c INFO: running cryptominisat-2.9.0@0 for 1649 with 4694 remaining (b = 0.49)
1958.45/494.91	c INFO: running SApperloT@0 for 97 with 4305 remaining (b = 0.03)
2358.17/595.07	c INFO: running lingeling-276@0 for 485 with 4205 remaining (b = 0.22)
4305.47/1082.66	c INFO: running precosat-570@0 for 388 with 3718 remaining (b = 0.25)
4800.29/1206.54	c INFO: running lingeling-276@0 for 97 with 3594 remaining (b = 0.08)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317867-1304270918/watcher-3317867-1304270918 -o /tmp/evaluation-result-3317867-1304270918/solver-3317867-1304270918 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317867-1304270918.cnf -seed 966247080 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.85 8.00 7.00 6/212 26122
/proc/meminfo: memFree=18598036/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=15380 CPUtime=0 cores=1,3,5,7
/proc/26122/stat : 26122 (python) R 26120 26122 25339 0 -1 4202496 792 0 0 0 0 0 0 0 25 0 1 0 538691568 15749120 679 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278877696 0 2147483391 16781312 2 0 0 0 17 3 0 0 0
/proc/26122/statm: 3845 679 322 364 0 332 0

[startup+0.099488 s]
/proc/loadavg: 7.85 8.00 7.00 6/212 26122
/proc/meminfo: memFree=18598036/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=87708 CPUtime=0.05 cores=1,3,5,7
/proc/26122/stat : 26122 (python) R 26120 26122 25339 0 -1 4202496 2636 0 0 0 4 1 0 0 25 0 1 0 538691568 89812992 2352 33554432000 4194304 5683644 140737455363456 18446744073709551615 237273917184 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/26122/statm: 21927 2354 717 364 0 1623 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 87708

[startup+0.100488 s]
/proc/loadavg: 7.85 8.00 7.00 6/212 26122
/proc/meminfo: memFree=18598036/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=87708 CPUtime=0.05 cores=1,3,5,7
/proc/26122/stat : 26122 (python) R 26120 26122 25339 0 -1 4202496 2639 0 0 0 4 1 0 0 25 0 1 0 538691568 89812992 2354 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278877696 0 2147483391 16781312 2 0 0 0 17 3 0 0 0
/proc/26122/statm: 21927 2354 717 364 0 1623 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 87708

[startup+0.300452 s]
/proc/loadavg: 7.85 8.00 7.00 6/212 26122
/proc/meminfo: memFree=18598036/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=135752 CPUtime=0.16 cores=1,3,5,7
/proc/26122/stat : 26122 (python) S 26120 26122 25339 0 -1 4202496 5961 936 1 0 13 3 0 0 25 0 1 0 538691568 139010048 4150 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278875653 0 2147483391 16781312 2 18446744071699199166 0 0 17 3 0 0 0
/proc/26122/statm: 33938 4150 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.16
Current children cumulated vsize (KiB) 135752

[startup+0.700379 s]
/proc/loadavg: 7.85 8.00 7.00 6/212 26122
/proc/meminfo: memFree=18598036/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=282360 CPUtime=0.42 cores=1,3,5,7
/proc/26122/stat : 26122 (python) R 26120 26122 25339 0 -1 4202496 10234 936 1 0 32 10 0 0 16 0 1 0 538691568 289136640 7916 33554432000 4194304 5683644 140737455363456 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/26122/statm: 70590 7917 1761 364 0 6514 0
Current children cumulated CPU time (s) 0.42
Current children cumulated vsize (KiB) 282360

[startup+1.50023 s]
/proc/loadavg: 7.85 8.00 7.00 6/217 26131
/proc/meminfo: memFree=18545792/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=306348 CPUtime=1.23 cores=1,3,5,7
/proc/26122/stat : 26122 (python) R 26120 26122 25339 0 -1 4202496 19615 936 1 0 111 12 0 0 16 0 1 0 538691568 313700352 13971 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278566152 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/26122/statm: 76587 13971 1826 364 0 12511 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 306348

[startup+3.10095 s]
/proc/loadavg: 7.85 8.00 7.00 6/217 26131
/proc/meminfo: memFree=18537608/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=306348 CPUtime=2.83 cores=1,3,5,7
/proc/26122/stat : 26122 (python) R 26120 26122 25339 0 -1 4202496 19615 936 1 0 271 12 0 0 18 0 1 0 538691568 313700352 13971 33554432000 4194304 5683644 140737455363456 18446744073709551615 46977953394944 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/26122/statm: 76587 13971 1826 364 0 12511 0
Current children cumulated CPU time (s) 2.83
Current children cumulated vsize (KiB) 306348

[startup+6.30039 s]
/proc/loadavg: 7.62 7.95 6.99 6/217 26131
/proc/meminfo: memFree=18535268/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=345132 CPUtime=5.64 cores=1,3,5,7
/proc/26122/stat : 26122 (python) S 26120 26122 25339 0 -1 4202496 22371 936 1 0 551 13 0 0 25 0 5 0 538691568 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/26122/statm: 86283 13479 1881 364 0 22192 0
Current children cumulated CPU time (s) 5.64
Current children cumulated vsize (KiB) 345132

[startup+12.7012 s]
/proc/loadavg: 7.65 7.95 6.99 9/229 26143
/proc/meminfo: memFree=18436760/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=345132 CPUtime=5.64 cores=1,3,5,7
/proc/26122/stat : 26122 (python) S 26120 26122 25339 0 -1 4202496 22371 936 1 0 551 13 0 0 25 0 5 0 538691568 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/26122/statm: 86283 13479 1881 364 0 22192 0
[pid=26132] ppid=26122 vsize=304120 CPUtime=0 cores=1,3,5,7
/proc/26132/stat : 26132 (python) S 26122 26122 25339 0 -1 4202560 985 0 0 0 0 0 0 0 18 0 1 0 538692152 311418880 11963 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899727 0 0 16781312 514 11074215148557140232 0 0 17 3 0 0 0
/proc/26132/statm: 76030 11963 381 364 0 11948 0
[pid=26134] ppid=26132 vsize=19104 CPUtime=6.8 cores=1,3,5,7
/proc/26134/stat : 26134 (lingeling) R 26132 26134 26134 0 -1 4202496 5339 0 0 0 680 0 0 0 25 0 1 0 538692153 19562496 3944 33554432000 4194304 4352236 140736969048912 18446744073709551615 4331465 0 0 16781312 17506 0 0 0 17 7 0 0 0
/proc/26134/statm: 4776 3944 128 39 0 3856 0
[pid=26135] ppid=26122 vsize=314376 CPUtime=0.01 cores=1,3,5,7
/proc/26135/stat : 26135 (python) S 26122 26122 25339 0 -1 4202560 898 0 0 0 1 0 0 0 15 0 1 0 538692155 321921024 11956 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 8244419664336093447 0 0 17 3 0 0 0
/proc/26135/statm: 78594 11956 370 364 0 14509 0
[pid=26137] ppid=26135 vsize=64276 CPUtime=6.82 cores=1,3,5,7
/proc/26137/stat : 26137 (cryptominisat-2) R 26135 26137 26137 0 -1 4202496 11398 0 0 0 680 2 0 0 25 0 1 0 538692155 65818624 8792 33554432000 4194304 4677096 140737328706368 18446744073709551615 4346584 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26137/statm: 16069 8792 392 118 0 11028 0
[pid=26138] ppid=26122 vsize=324632 CPUtime=0 cores=1,3,5,7
/proc/26138/stat : 26138 (python) S 26122 26122 25339 0 -1 4202560 896 0 0 0 0 0 0 0 18 0 1 0 538692157 332423168 11960 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 4551859945787588872 0 0 17 3 0 0 0
/proc/26138/statm: 81158 11960 370 364 0 17070 0
[pid=26140] ppid=26138 vsize=19104 CPUtime=6.76 cores=1,3,5,7
/proc/26140/stat : 26140 (lingeling) R 26138 26140 26140 0 -1 4202496 5340 0 0 0 676 0 0 0 25 0 1 0 538692158 19562496 3944 33554432000 4194304 4352236 140735121895920 18446744073709551615 4328194 0 0 16781312 17506 0 0 0 17 3 0 0 0
/proc/26140/statm: 4776 3944 128 39 0 3856 0
[pid=26141] ppid=26122 vsize=334888 CPUtime=0 cores=1,3,5,7
/proc/26141/stat : 26141 (python) S 26122 26122 25339 0 -1 4202560 927 0 0 0 0 0 0 0 18 0 1 0 538692160 342925312 11964 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 4571755883570102535 0 0 17 5 0 0 0
/proc/26141/statm: 83722 11964 370 364 0 19631 0
[pid=26143] ppid=26141 vsize=19108 CPUtime=6.77 cores=1,3,5,7
/proc/26143/stat : 26143 (lingeling) R 26141 26143 26143 0 -1 4202496 5342 0 0 0 676 1 0 0 25 0 1 0 538692160 19566592 3946 33554432000 4194304 4352236 140734416209952 18446744073709551615 4331666 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/26143/statm: 4777 3946 128 39 0 3857 0
[pid=26122/tid=26133] ppid=26120 vsize=345132 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26133/stat : 26133 (python) S 26120 26122 25339 0 -1 4202560 25 936 0 0 0 0 0 0 25 0 5 0 538692153 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 7 0 0 0
[pid=26122/tid=26136] ppid=26120 vsize=345132 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26136/stat : 26136 (python) S 26120 26122 25339 0 -1 4202560 19 936 0 0 0 0 0 0 25 0 5 0 538692155 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 3 0 0 0
[pid=26122/tid=26139] ppid=26120 vsize=345132 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26139/stat : 26139 (python) S 26120 26122 25339 0 -1 4202560 21 936 0 0 0 0 0 0 25 0 5 0 538692158 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 3 0 0 0
[pid=26122/tid=26142] ppid=26120 vsize=345132 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26142/stat : 26142 (python) S 26120 26122 25339 0 -1 4202560 29 936 0 0 0 0 0 0 25 0 5 0 538692160 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 32.8
Current children cumulated vsize (KiB) 1744740

[startup+25.501 s]
/proc/loadavg: 7.73 7.95 7.01 9/229 26144
/proc/meminfo: memFree=18404124/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=345132 CPUtime=5.64 cores=1,3,5,7
/proc/26122/stat : 26122 (python) S 26120 26122 25339 0 -1 4202496 22371 936 1 0 551 13 0 0 25 0 5 0 538691568 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/26122/statm: 86283 13479 1881 364 0 22192 0
[pid=26132] ppid=26122 vsize=304120 CPUtime=0.02 cores=1,3,5,7
/proc/26132/stat : 26132 (python) S 26122 26122 25339 0 -1 4202560 997 0 0 0 1 1 0 0 18 0 1 0 538692152 311418880 11963 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899727 0 0 16781312 514 16514347989846688008 0 0 17 3 0 0 0
/proc/26132/statm: 76030 11963 381 364 0 11948 0
[pid=26134] ppid=26132 vsize=19104 CPUtime=19.52 cores=1,3,5,7
/proc/26134/stat : 26134 (lingeling) R 26132 26134 26134 0 -1 4202496 5339 0 0 0 1952 0 0 0 25 0 1 0 538692153 19562496 3944 33554432000 4194304 4352236 140736969048912 18446744073709551615 4328432 0 0 16781312 17506 0 0 0 17 7 0 0 0
/proc/26134/statm: 4776 3944 128 39 0 3856 0
[pid=26135] ppid=26122 vsize=314376 CPUtime=0.03 cores=1,3,5,7
/proc/26135/stat : 26135 (python) S 26122 26122 25339 0 -1 4202560 912 0 0 0 2 1 0 0 15 0 1 0 538692155 321921024 11956 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 8244419664336093447 0 0 17 3 0 0 0
/proc/26135/statm: 78594 11956 370 364 0 14509 0
[pid=26137] ppid=26135 vsize=119084 CPUtime=19.62 cores=1,3,5,7
/proc/26137/stat : 26137 (cryptominisat-2) R 26135 26137 26137 0 -1 4202496 17703 0 0 0 1958 4 0 0 25 0 1 0 538692155 121942016 15097 33554432000 4194304 4677096 140737328706368 18446744073709551615 4319069 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26137/statm: 29771 15097 393 118 0 24730 0
[pid=26138] ppid=26122 vsize=324632 CPUtime=0.02 cores=1,3,5,7
/proc/26138/stat : 26138 (python) S 26122 26122 25339 0 -1 4202560 904 0 0 0 1 1 0 0 18 0 1 0 538692157 332423168 11960 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688008 0 0 17 3 0 0 0
/proc/26138/statm: 81158 11960 370 364 0 17070 0
[pid=26140] ppid=26138 vsize=19104 CPUtime=19.5 cores=1,3,5,7
/proc/26140/stat : 26140 (lingeling) R 26138 26140 26140 0 -1 4202496 5340 0 0 0 1950 0 0 0 25 0 1 0 538692158 19562496 3944 33554432000 4194304 4352236 140735121895920 18446744073709551615 4331514 0 0 16781312 17506 0 0 0 17 3 0 0 0
/proc/26140/statm: 4776 3944 128 39 0 3856 0
[pid=26141] ppid=26122 vsize=334888 CPUtime=0.02 cores=1,3,5,7
/proc/26141/stat : 26141 (python) S 26122 26122 25339 0 -1 4202560 933 0 0 0 1 1 0 0 18 0 1 0 538692160 342925312 11964 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688007 0 0 17 5 0 0 0
/proc/26141/statm: 83722 11964 370 364 0 19631 0
[pid=26143] ppid=26141 vsize=19108 CPUtime=19.55 cores=1,3,5,7
/proc/26143/stat : 26143 (lingeling) R 26141 26143 26143 0 -1 4202496 5342 0 0 0 1954 1 0 0 25 0 1 0 538692160 19566592 3946 33554432000 4194304 4352236 140734416209952 18446744073709551615 4207584 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/26143/statm: 4777 3946 128 39 0 3857 0
[pid=26122/tid=26133] ppid=26120 vsize=345132 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26133/stat : 26133 (python) S 26120 26122 25339 0 -1 4202560 25 936 0 0 0 0 0 0 25 0 5 0 538692153 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 7 0 0 0
[pid=26122/tid=26136] ppid=26120 vsize=345132 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26136/stat : 26136 (python) S 26120 26122 25339 0 -1 4202560 19 936 0 0 0 0 0 0 25 0 5 0 538692155 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 3 0 0 0
[pid=26122/tid=26139] ppid=26120 vsize=345132 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26139/stat : 26139 (python) S 26120 26122 25339 0 -1 4202560 21 936 0 0 0 0 0 0 25 0 5 0 538692158 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 3 0 0 0
[pid=26122/tid=26142] ppid=26120 vsize=345132 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26142/stat : 26142 (python) S 26120 26122 25339 0 -1 4202560 29 936 0 0 0 0 0 0 25 0 5 0 538692160 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 83.92
Current children cumulated vsize (KiB) 1799548

[startup+51.1014 s]
/proc/loadavg: 7.96 7.99 7.04 9/229 26145
/proc/meminfo: memFree=18303844/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=345132 CPUtime=5.64 cores=1,3,5,7
/proc/26122/stat : 26122 (python) S 26120 26122 25339 0 -1 4202496 22371 936 1 0 551 13 0 0 25 0 5 0 538691568 353415168 13479 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/26122/statm: 86283 13479 1881 364 0 22192 0
[pid=26132] ppid=26122 vsize=304120 CPUtime=0.05 cores=1,3,5,7
/proc/26132/stat : 26132 (python) S 26122 26122 25339 0 -1 4202560 1011 0 0 0 2 3 0 0 18 0 1 0 538692152 311418880 11963 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899727 0 0 16781312 514 16514347989846688008 0 0 17 3 0 0 0
/proc/26132/statm: 76030 11963 381 364 0 11948 0
[pid=26134] ppid=26132 vsize=22068 CPUtime=44.96 cores=1,3,5,7
/proc/26134/stat : 26134 (lingeling) R 26132 26134 26134 0 -1 4202496 6803 0 0 0 4496 0 0 0 25 0 1 0 538692153 22597632 4558 33554432000 4194304 4352236 140736969048912 18446744073709551615 4281188 0 0 16781312 17506 0 0 0 17 7 0 0 0
/proc/26134/statm: 5517 4558 128 39 0 4597 0
[pid=26135] ppid=26122 vsize=314376 CPUtime=0.08 cores=1,3,5,7
/proc/26135/stat : 26135 (python) S 26122 26122 25339 0 -1 4202560 922 0 0 0 5 3 0 0 15 0 1 0 538692155 321921024 11956 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688007 0 0 17 3 0 0 0
/proc/26135/statm: 78594 11956 370 364 0 14509 0
[pid=26137] ppid=26135 vsize=231988 CPUtime=45.22 cores=1,3,5,7
/proc/26137/stat : 26137 (cryptominisat-2) R 26135 26137 26137 0 -1 4202496 47896 0 0 0 4511 11 0 0 25 0 1 0 538692155 237555712 38003 33554432000 4194304 4677096 140737328706368 18446744073709551615 4319826 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26137/statm: 57997 38003 398 118 0 52956 0
[pid=26138] ppid=26122 vsize=324632 CPUtime=0.06 cores=1,3,5,7
/proc/26138/stat : 26138 (python) S 26122 26122 25339 0 -1 4202560 916 0 0 0 3 3 0 0 18 0 1 0 538692157 332423168 11960 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688008 0 0 17 3 0 0 0
/proc/26138/statm: 81158 11960 370 364 0 17070 0
[pid=26140] ppid=26138 vsize=20264 CPUtime=44.98 cores=1,3,5,7
/proc/26140/stat : 26140 (lingeling) R 26138 26140 26140 0 -1 4202496 6478 0 0 0 4497 1 0 0 25 0 1 0 538692158 20750336 4234 33554432000 4194304 4352236 140735121895920 18446744073709551615 4207874 0 0 16781312 17506 0 0 0 17 3 0 0 0
/proc/26140/statm: 5066 4234 128 39 0 4146 0
[pid=26141] ppid=26122 vsize=334888 CPUtime=0.06 cores=1,3,5,7
/proc/26141/stat : 26141 (python) S 26122 26122 25339 0 -1 4202560 944 0 0 0 3 3 0 0 18 0 1 0 538692160 342925312 11964 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688007 0 0 17 5 0 0 0
/proc/26141/statm: 83722 11964 370 364 0 19631 0

################
# More data... #
################

/proc/26122/task/26169/stat : 26169 (python) S 26120 26122 25339 0 -1 4202560 28 936 0 0 0 0 0 0 20 0 8 0 538741060 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
[pid=26122/tid=26211] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26211/stat : 26211 (python) S 26120 26122 25339 0 -1 4202560 33 936 0 0 0 0 0 0 20 0 8 0 538799835 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4703.62
Current children cumulated vsize (KiB) 4248080



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1206.5 s]
/proc/loadavg: 5.02 5.35 6.18 6/215 26215
/proc/meminfo: memFree=17534272/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=375900 CPUtime=5.73 cores=1,3,5,7
/proc/26122/stat : 26122 (python) S 26120 26122 25339 0 -1 4202496 24917 936 1 0 560 13 0 0 19 0 8 0 538691568 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/26122/statm: 93975 13498 1888 364 0 29875 0
[pid=26132] ppid=26122 vsize=314364 CPUtime=0.8 cores=1,3,5,7
/proc/26132/stat : 26132 (python) S 26122 26122 25339 0 -1 4202560 1224 0 0 0 48 32 0 0 18 0 2 0 538692152 321908736 12002 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/26132/statm: 78591 12002 415 364 0 14509 0
[pid=26134] ppid=26132 vsize=35640 CPUtime=583.96 cores=1,3,5,7
/proc/26134/stat : 26134 (lingeling) T 26132 26134 26134 0 -1 4202496 18611 0 0 0 58394 2 0 0 25 0 1 0 538692153 36495360 8077 33554432000 4194304 4352236 140736969048912 18446744073709551615 4329000 0 0 16781312 17506 18446744071562691360 0 0 17 7 0 0 0
/proc/26134/statm: 8910 8077 128 39 0 7990 0
[pid=26132/tid=26147] ppid=26122 vsize=314364 CPUtime=0 cores=1,3,5,7
/proc/26132/task/26147/stat : 26147 (python) S 26122 26122 25339 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 538702167 321908736 12002 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 514 18446604459786077184 0 0 -1 3 0 0 0
[pid=26135] ppid=26122 vsize=314376 CPUtime=0.77 cores=1,3,5,7
/proc/26135/stat : 26135 (python) S 26122 26122 25339 0 -1 4202560 1131 0 0 0 40 37 0 0 16 0 2 0 538692155 321921024 11986 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/26135/statm: 78594 11986 399 364 0 14509 0
[pid=26137] ppid=26135 vsize=588000 CPUtime=489.03 cores=1,3,5,7
/proc/26137/stat : 26137 (cryptominisat-2) T 26135 26137 26137 0 -1 4202496 296716 0 0 0 48814 89 0 0 25 0 1 0 538692155 602112000 115768 33554432000 4194304 4677096 140737328706368 18446744073709551615 4319417 0 0 16781312 2 18446744071562691360 0 0 17 1 0 0 0
/proc/26137/statm: 147000 115768 393 118 0 141959 0
[pid=26135/tid=26167] ppid=26122 vsize=314376 CPUtime=0 cores=1,3,5,7
/proc/26135/task/26167/stat : 26167 (python) S 26122 26122 25339 0 -1 4202560 49 0 0 0 0 0 0 0 16 0 2 0 538741058 321921024 11986 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 514 18446604466862041792 0 0 -1 3 0 0 0
[pid=26138] ppid=26122 vsize=324632 CPUtime=1.68 cores=1,3,5,7
/proc/26138/stat : 26138 (python) S 26122 26122 25339 0 -1 4202560 969 0 0 0 95 73 0 0 18 0 1 0 538692157 332423168 11960 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688008 0 0 17 3 0 0 0
/proc/26138/statm: 81158 11960 370 364 0 17070 0
[pid=26140] ppid=26138 vsize=53104 CPUtime=1197.35 cores=1,3,5,7
/proc/26140/stat : 26140 (lingeling) R 26138 26140 26140 0 -1 4202496 60055 0 0 0 119727 8 0 0 25 0 1 0 538692158 54378496 12444 33554432000 4194304 4352236 140735121895920 18446744073709551615 4231316 0 0 16781312 17506 0 0 0 17 3 0 0 0
/proc/26140/statm: 13276 12444 128 39 0 12356 0
[pid=26141] ppid=26122 vsize=334888 CPUtime=1.68 cores=1,3,5,7
/proc/26141/stat : 26141 (python) S 26122 26122 25339 0 -1 4202560 999 0 0 0 90 78 0 0 18 0 1 0 538692160 342925312 11964 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688007 0 0 17 5 0 0 0
/proc/26141/statm: 83722 11964 370 364 0 19631 0
[pid=26143] ppid=26141 vsize=48172 CPUtime=1195.07 cores=1,3,5,7
/proc/26143/stat : 26143 (lingeling) R 26141 26143 26143 0 -1 4202496 58834 0 0 0 119500 7 0 0 25 0 1 0 538692160 49328128 11212 33554432000 4194304 4352236 140734416209952 18446744073709551615 4278712 0 0 16781312 17506 0 0 0 17 7 0 0 0
/proc/26143/statm: 12043 11212 128 39 0 11123 0
[pid=26148] ppid=26122 vsize=345144 CPUtime=1.59 cores=1,3,5,7
/proc/26148/stat : 26148 (python) S 26122 26122 25339 0 -1 4202560 1024 0 0 0 86 73 0 0 15 0 1 0 538702169 353427456 11968 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688005 0 0 17 5 0 0 0
/proc/26148/statm: 86286 11968 370 364 0 22192 0
[pid=26150] ppid=26148 vsize=526276 CPUtime=1099.31 cores=1,3,5,7
/proc/26150/stat : 26150 (cryptominisat-2) R 26148 26150 26150 0 -1 4202496 675512 0 0 0 109715 216 0 0 25 0 1 0 538702169 538906624 117803 33554432000 4194304 4677096 140737173485872 18446744073709551615 4320006 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26150/statm: 131569 117803 393 118 0 126528 0
[pid=26168] ppid=26122 vsize=355400 CPUtime=0.14 cores=1,3,5,7
/proc/26168/stat : 26168 (python) S 26122 26122 25339 0 -1 4202560 1127 0 0 0 7 7 0 0 19 0 2 0 538741060 363929600 12002 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/26168/statm: 88850 12002 399 364 0 24753 0
[pid=26170] ppid=26168 vsize=98716 CPUtime=99.69 cores=1,3,5,7
/proc/26170/stat : 26170 (SApperloT-base) T 26168 26170 26170 0 -1 4202496 24585 0 0 0 9963 6 0 0 25 0 1 0 538741061 101085184 24222 33554432000 134512640 135674940 4292892432 18446744073709551615 134564250 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/26170/statm: 24679 24222 100 284 0 24393 0
[pid=26168/tid=26196] ppid=26122 vsize=355400 CPUtime=0 cores=1,3,5,7
/proc/26168/task/26196/stat : 26196 (python) S 26122 26122 25339 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538751073 363929600 12002 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 514 18446604459217454400 0 0 -1 5 0 0 0
[pid=26210] ppid=26122 vsize=365656 CPUtime=0.16 cores=1,3,5,7
/proc/26210/stat : 26210 (python) S 26122 26122 25339 0 -1 4202560 968 0 0 0 8 8 0 0 18 0 1 0 538799835 374431744 11976 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688008 0 0 17 5 0 0 0
/proc/26210/statm: 91414 11976 370 364 0 27314 0
[pid=26212] ppid=26210 vsize=40400 CPUtime=123.33 cores=1,3,5,7
/proc/26212/stat : 26212 (precosat) R 26210 26212 26212 0 -1 4202496 10537 0 0 0 12332 1 0 0 25 0 1 0 538799835 41369600 9829 33554432000 4194304 4890832 140734196399536 18446744073709551615 4238327 0 0 16781312 17442 0 0 0 17 5 0 0 0
/proc/26212/statm: 10100 9829 86 171 0 9927 0
[pid=26122/tid=26133] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26133/stat : 26133 (python) S 26120 26122 25339 0 -1 4202560 30 936 0 0 0 0 0 0 18 0 8 0 538692153 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 7 0 0 0
[pid=26122/tid=26136] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26136/stat : 26136 (python) S 26120 26122 25339 0 -1 4202560 19 936 0 0 0 0 0 0 25 0 8 0 538692155 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 3 0 0 0
[pid=26122/tid=26139] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26139/stat : 26139 (python) S 26120 26122 25339 0 -1 4202560 21 936 0 0 0 0 0 0 25 0 8 0 538692158 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 3 0 0 0
[pid=26122/tid=26142] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26142/stat : 26142 (python) S 26120 26122 25339 0 -1 4202560 29 936 0 0 0 0 0 0 25 0 8 0 538692160 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
[pid=26122/tid=26149] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26149/stat : 26149 (python) S 26120 26122 25339 0 -1 4202560 30 936 0 0 0 0 0 0 20 0 8 0 538702169 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
[pid=26122/tid=26169] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26169/stat : 26169 (python) S 26120 26122 25339 0 -1 4202560 28 936 0 0 0 0 0 0 20 0 8 0 538741060 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
[pid=26122/tid=26211] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26211/stat : 26211 (python) S 26120 26122 25339 0 -1 4202560 33 936 0 0 0 0 0 0 20 0 8 0 538799835 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 4120668

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1206.5 s]
/proc/loadavg: 5.02 5.35 6.18 6/215 26215
/proc/meminfo: memFree=17534272/32950904 swapFree=67082284/67111528
[pid=26122] ppid=26120 vsize=375900 CPUtime=5.73 cores=1,3,5,7
/proc/26122/stat : 26122 (python) S 26120 26122 25339 0 -1 4202496 24917 936 1 0 560 13 0 0 19 0 8 0 538691568 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/26122/statm: 93975 13498 1888 364 0 29875 0
[pid=26132] ppid=26122 vsize=314364 CPUtime=0.8 cores=1,3,5,7
/proc/26132/stat : 26132 (python) S 26122 26122 25339 0 -1 4202560 1224 0 0 0 48 32 0 0 18 0 2 0 538692152 321908736 12002 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/26132/statm: 78591 12002 415 364 0 14509 0
[pid=26134] ppid=26132 vsize=35640 CPUtime=583.96 cores=1,3,5,7
/proc/26134/stat : 26134 (lingeling) T 26132 26134 26134 0 -1 4202496 18611 0 0 0 58394 2 0 0 25 0 1 0 538692153 36495360 8077 33554432000 4194304 4352236 140736969048912 18446744073709551615 4329000 0 0 16781312 17506 18446744071562691360 0 0 17 7 0 0 0
/proc/26134/statm: 8910 8077 128 39 0 7990 0
[pid=26132/tid=26147] ppid=26122 vsize=314364 CPUtime=0 cores=1,3,5,7
/proc/26132/task/26147/stat : 26147 (python) S 26122 26122 25339 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 538702167 321908736 12002 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 514 18446604459786077184 0 0 -1 3 0 0 0
[pid=26135] ppid=26122 vsize=314376 CPUtime=0.77 cores=1,3,5,7
/proc/26135/stat : 26135 (python) S 26122 26122 25339 0 -1 4202560 1131 0 0 0 40 37 0 0 16 0 2 0 538692155 321921024 11986 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/26135/statm: 78594 11986 399 364 0 14509 0
[pid=26137] ppid=26135 vsize=588000 CPUtime=489.03 cores=1,3,5,7
/proc/26137/stat : 26137 (cryptominisat-2) T 26135 26137 26137 0 -1 4202496 296716 0 0 0 48814 89 0 0 25 0 1 0 538692155 602112000 115768 33554432000 4194304 4677096 140737328706368 18446744073709551615 4319417 0 0 16781312 2 18446744071562691360 0 0 17 1 0 0 0
/proc/26137/statm: 147000 115768 393 118 0 141959 0
[pid=26135/tid=26167] ppid=26122 vsize=314376 CPUtime=0 cores=1,3,5,7
/proc/26135/task/26167/stat : 26167 (python) S 26122 26122 25339 0 -1 4202560 49 0 0 0 0 0 0 0 16 0 2 0 538741058 321921024 11986 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 514 18446604466862041792 0 0 -1 3 0 0 0
[pid=26138] ppid=26122 vsize=324632 CPUtime=1.68 cores=1,3,5,7
/proc/26138/stat : 26138 (python) S 26122 26122 25339 0 -1 4202560 969 0 0 0 95 73 0 0 18 0 1 0 538692157 332423168 11960 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688008 0 0 17 3 0 0 0
/proc/26138/statm: 81158 11960 370 364 0 17070 0
[pid=26140] ppid=26138 vsize=53104 CPUtime=1197.35 cores=1,3,5,7
/proc/26140/stat : 26140 (lingeling) R 26138 26140 26140 0 -1 4202496 60055 0 0 0 119727 8 0 0 25 0 1 0 538692158 54378496 12444 33554432000 4194304 4352236 140735121895920 18446744073709551615 4231316 0 0 16781312 17506 0 0 0 17 3 0 0 0
/proc/26140/statm: 13276 12444 128 39 0 12356 0
[pid=26141] ppid=26122 vsize=334888 CPUtime=1.68 cores=1,3,5,7
/proc/26141/stat : 26141 (python) S 26122 26122 25339 0 -1 4202560 999 0 0 0 90 78 0 0 18 0 1 0 538692160 342925312 11964 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688007 0 0 17 5 0 0 0
/proc/26141/statm: 83722 11964 370 364 0 19631 0
[pid=26143] ppid=26141 vsize=48172 CPUtime=1195.07 cores=1,3,5,7
/proc/26143/stat : 26143 (lingeling) R 26141 26143 26143 0 -1 4202496 58834 0 0 0 119500 7 0 0 25 0 1 0 538692160 49328128 11212 33554432000 4194304 4352236 140734416209952 18446744073709551615 4278712 0 0 16781312 17506 0 0 0 17 7 0 0 0
/proc/26143/statm: 12043 11212 128 39 0 11123 0
[pid=26148] ppid=26122 vsize=345144 CPUtime=1.59 cores=1,3,5,7
/proc/26148/stat : 26148 (python) S 26122 26122 25339 0 -1 4202560 1024 0 0 0 86 73 0 0 15 0 1 0 538702169 353427456 11968 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688005 0 0 17 5 0 0 0
/proc/26148/statm: 86286 11968 370 364 0 22192 0
[pid=26150] ppid=26148 vsize=526276 CPUtime=1099.31 cores=1,3,5,7
/proc/26150/stat : 26150 (cryptominisat-2) R 26148 26150 26150 0 -1 4202496 675512 0 0 0 109715 216 0 0 25 0 1 0 538702169 538906624 117803 33554432000 4194304 4677096 140737173485872 18446744073709551615 4320006 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/26150/statm: 131569 117803 393 118 0 126528 0
[pid=26168] ppid=26122 vsize=355400 CPUtime=0.14 cores=1,3,5,7
/proc/26168/stat : 26168 (python) S 26122 26122 25339 0 -1 4202560 1127 0 0 0 7 7 0 0 19 0 2 0 538741060 363929600 12002 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/26168/statm: 88850 12002 399 364 0 24753 0
[pid=26170] ppid=26168 vsize=98716 CPUtime=99.69 cores=1,3,5,7
/proc/26170/stat : 26170 (SApperloT-base) T 26168 26170 26170 0 -1 4202496 24585 0 0 0 9963 6 0 0 25 0 1 0 538741061 101085184 24222 33554432000 134512640 135674940 4292892432 18446744073709551615 134564250 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/26170/statm: 24679 24222 100 284 0 24393 0
[pid=26168/tid=26196] ppid=26122 vsize=355400 CPUtime=0 cores=1,3,5,7
/proc/26168/task/26196/stat : 26196 (python) S 26122 26122 25339 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538751073 363929600 12002 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 514 18446604459217454400 0 0 -1 5 0 0 0
[pid=26210] ppid=26122 vsize=365656 CPUtime=0.16 cores=1,3,5,7
/proc/26210/stat : 26210 (python) S 26122 26122 25339 0 -1 4202560 968 0 0 0 8 8 0 0 18 0 1 0 538799835 374431744 11976 33554432000 4194304 5683644 140737455363456 18446744073709551615 237278899782 0 0 16781312 514 16514347989846688008 0 0 17 5 0 0 0
/proc/26210/statm: 91414 11976 370 364 0 27314 0
[pid=26212] ppid=26210 vsize=40400 CPUtime=123.33 cores=1,3,5,7
/proc/26212/stat : 26212 (precosat) R 26210 26212 26212 0 -1 4202496 10537 0 0 0 12332 1 0 0 25 0 1 0 538799835 41369600 9829 33554432000 4194304 4890832 140734196399536 18446744073709551615 4238327 0 0 16781312 17442 0 0 0 17 5 0 0 0
/proc/26212/statm: 10100 9829 86 171 0 9927 0
[pid=26122/tid=26133] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26133/stat : 26133 (python) S 26120 26122 25339 0 -1 4202560 30 936 0 0 0 0 0 0 18 0 8 0 538692153 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 7 0 0 0
[pid=26122/tid=26136] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26136/stat : 26136 (python) S 26120 26122 25339 0 -1 4202560 19 936 0 0 0 0 0 0 25 0 8 0 538692155 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 3 0 0 0
[pid=26122/tid=26139] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26139/stat : 26139 (python) S 26120 26122 25339 0 -1 4202560 21 936 0 0 0 0 0 0 25 0 8 0 538692158 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 3 0 0 0
[pid=26122/tid=26142] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26142/stat : 26142 (python) S 26120 26122 25339 0 -1 4202560 29 936 0 0 0 0 0 0 25 0 8 0 538692160 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
[pid=26122/tid=26149] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26149/stat : 26149 (python) S 26120 26122 25339 0 -1 4202560 30 936 0 0 0 0 0 0 20 0 8 0 538702169 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
[pid=26122/tid=26169] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26169/stat : 26169 (python) S 26120 26122 25339 0 -1 4202560 28 936 0 0 0 0 0 0 20 0 8 0 538741060 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
[pid=26122/tid=26211] ppid=26120 vsize=375900 CPUtime=0 cores=1,3,5,7
/proc/26122/task/26211/stat : 26211 (python) S 26120 26122 25339 0 -1 4202560 33 936 0 0 0 0 0 0 20 0 8 0 538799835 384921600 13498 33554432000 4194304 5683644 140737455363456 18446744073709551615 237290703281 0 0 16781312 2 18446604459217456192 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 4120668

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 26122 and gives
#  childrusage.ru_utime.tv_sec=1204
#  childrusage.ru_utime.tv_usec=419900
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=350490
# CPU time returned by wait4() is 1207.77
# while last known CPU time is 4800.29
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1206.55
CPU time (s): 4800.29
CPU user time (s): 4793.79
CPU system time (s): 6.5
CPU usage (%): 397.852
Max. virtual memory (cumulated for all children) (KiB): 4253952

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1204.42
system time used= 3.35049
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 92987
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8371
involuntary context switches= 9754

runsolver used 2.26166 second user time and 5.60515 second system time

The end

Launcher Data 

Begin job on node121 at 2011-05-01 19:28:38
IDJOB=3317867
IDBENCH=84223
IDSOLVER=1706
FILE ID=node121/3317867-1304270918
RUNJOBID= node121-1304267283-25358
PBS_JOBID= 13173958
Free space on /tmp= 73572 MiB

SOLVER NAME= borg-sat 11.03.02-industrial
BENCH NAME= SAT03/industrial/li/li-exam/li-exam-61.shuffled-as.sat03-366.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317867-1304270918/watcher-3317867-1304270918 -o /tmp/evaluation-result-3317867-1304270918/solver-3317867-1304270918 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317867-1304270918.cnf -seed 966247080 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3be662940f394fa876137a905247d755
RANDOM SEED=966247080

node121.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950904 kB
MemFree:      18598316 kB
Buffers:       2194028 kB
Cached:        9524988 kB
SwapCached:       2776 kB
Active:       10777628 kB
Inactive:      3049076 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950904 kB
LowFree:      18598316 kB
SwapTotal:    67111528 kB
SwapFree:     67082284 kB
Dirty:           34236 kB
Writeback:           0 kB
AnonPages:     2105288 kB
Mapped:          22968 kB
Slab:           452604 kB
PageTables:      13352 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586980 kB
Committed_AS:  2787892 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73564 MiB
End job on node121 at 2011-05-01 19:48:47