Trace number 3317914

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-industrial? (TO) 4800.15 1244.76

General information on the benchmark

Nameapplication/jarvisalo/HWMCC10-combination-timeframe-expansion/
hwmcc10-timeframe-expansion-k50-pdtvisns3p00-tseitin.cnf
MD5SUMa8ad0173824ed9404a8c153da07c5f9c
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark938.798
Satisfiable
(Un)Satisfiability was proved
Number of variables183325
Number of clauses546914
Sum of the clauses size1276130
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2364608
Number of clauses of size 3182304
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.38/0.64	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.38/0.64	c INFO: solving HOME/instance-3317914-1304269670.cnf
0.38/0.64	c INFO: computing task features
2.07/2.39	c INFO: computed variable-clause graph statistics
2.17/2.45	c INFO: computed clause balance statistics
10.57/10.87	c INFO: computed variable balance statistics
16.87/17.10	c INFO: computed small-clause counts
16.97/17.21	c INFO: computed Horn-clause counts
55.57/55.90	c INFO: computed variable graph statistics
55.57/55.90	c INFO: collected features for HOME/instance-3317914-1304269670.cnf in 55.21 s
55.67/55.92	c INFO: running lingeling-276@0 for 97 with 4744 remaining (b = 0.54)
55.67/55.94	c INFO: running cryptominisat-2.9.0@0 for 582 with 4744 remaining (b = 0.45)
55.67/55.97	c INFO: running lingeling-276@0 for 1261 with 4744 remaining (b = 0.52)
55.67/56.00	c INFO: running SApperloT@0 for 97 with 4744 remaining (b = 0.04)
455.21/156.07	c INFO: running lingeling-276@0 for 776 with 4644 remaining (b = 0.36)
455.65/156.17	c INFO: running cryptominisat-2.9.0@0 for 679 with 4644 remaining (b = 0.46)
2397.37/642.34	c INFO: running precosat-570@0 for 388 with 4158 remaining (b = 0.35)
3184.86/839.51	c INFO: running cryptominisat-2.9.0@0 for 97 with 3961 remaining (b = 0.11)
3570.17/936.01	c INFO: running lingeling-276@99 for 97 with 3864 remaining (b = 0.10)
3595.28/942.37	c INFO: running lingeling-276@0 for 291 with 3858 remaining (b = 0.16)
3949.09/1031.16	c INFO: running cryptominisat-2.9.0@0 for 388 with 3769 remaining (b = 0.15)
3959.85/1033.89	c INFO: running precosat-570@681 for 97 with 3766 remaining (b = 0.06)
4339.16/1129.04	c INFO: running lingeling-276@201 for 97 with 3671 remaining (b = 0.05)
4726.02/1226.20	c INFO: running precosat-570@775 for 97 with 3574 remaining (b = 0.06)
4780.98/1239.90	c INFO: running precosat-570@0 for 291 with 3560 remaining (b = 0.12)
4800.15/1244.75	c INFO: running cryptominisat-2.9.0@0 for 582 with 3555 remaining (b = 0.19)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317914-1304269670/watcher-3317914-1304269670 -o /tmp/evaluation-result-3317914-1304269670/solver-3317914-1304269670 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317914-1304269670.cnf -seed 824782661 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.76 4.22 4.23 6/198 25917
/proc/meminfo: memFree=19854408/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=15184 CPUtime=0 cores=1,3,5,7
/proc/25917/stat : 25917 (python) R 25915 25917 25339 0 -1 4202496 725 0 0 0 0 0 0 0 20 0 1 0 538566842 15548416 615 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278877696 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25917/statm: 3796 615 321 364 0 283 0

[startup+0.0349489 s]
/proc/loadavg: 5.76 4.22 4.23 6/198 25917
/proc/meminfo: memFree=19854408/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=16816 CPUtime=0.01 cores=1,3,5,7
/proc/25917/stat : 25917 (python) S 25915 25917 25339 0 -1 4202496 1214 0 0 0 1 0 0 0 20 0 1 0 538566842 17219584 1072 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278877696 0 2147483391 16781312 2 18446744071699199166 0 0 17 5 0 0 0
/proc/25917/statm: 4204 1072 417 364 0 691 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16816

[startup+0.100937 s]
/proc/loadavg: 5.76 4.22 4.23 6/198 25917
/proc/meminfo: memFree=19854408/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=85624 CPUtime=0.06 cores=1,3,5,7
/proc/25917/stat : 25917 (python) R 25915 25917 25339 0 -1 4202496 2631 0 0 0 5 1 0 0 17 0 1 0 538566842 87678976 2348 33554432000 4194304 5683644 140736575542208 18446744073709551615 237273955975 0 2147483391 16781312 2 0 0 0 17 5 0 0 0
/proc/25917/statm: 21406 2348 713 364 0 1623 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85624

[startup+0.300976 s]
/proc/loadavg: 5.76 4.22 4.23 6/198 25917
/proc/meminfo: memFree=19854408/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=135752 CPUtime=0.19 cores=1,3,5,7
/proc/25917/stat : 25917 (python) S 25915 25917 25339 0 -1 4202496 5963 935 1 0 14 5 0 0 17 0 1 0 538566842 139010048 4150 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278875653 0 2147483391 16781312 2 18446744071699199166 0 0 17 5 0 0 0
/proc/25917/statm: 33938 4150 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135752

[startup+0.700892 s]
/proc/loadavg: 5.76 4.22 4.23 6/198 25917
/proc/meminfo: memFree=19854408/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=291040 CPUtime=0.47 cores=1,3,5,7
/proc/25917/stat : 25917 (python) R 25915 25917 25339 0 -1 4202496 12500 935 1 0 35 12 0 0 15 0 1 0 538566842 298024960 10069 33554432000 4194304 5683644 140736575542208 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25917/statm: 72760 10071 1761 364 0 8684 0
Current children cumulated CPU time (s) 0.47
Current children cumulated vsize (KiB) 291040

[startup+1.50075 s]
/proc/loadavg: 5.76 4.22 4.23 6/199 25922
/proc/meminfo: memFree=19798972/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=344128 CPUtime=1.27 cores=1,3,5,7
/proc/25917/stat : 25917 (python) R 25915 25917 25339 0 -1 4202496 25558 935 1 0 114 13 0 0 16 0 1 0 538566842 352387072 23127 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278540485 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25917/statm: 86032 23127 1761 364 0 21956 0
Current children cumulated CPU time (s) 1.27
Current children cumulated vsize (KiB) 344128

[startup+3.1004 s]
/proc/loadavg: 5.70 4.24 4.23 6/199 25922
/proc/meminfo: memFree=19754456/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=389548 CPUtime=2.87 cores=1,3,5,7
/proc/25917/stat : 25917 (python) R 25915 25917 25339 0 -1 4202496 49559 935 1 0 271 16 0 0 18 0 1 0 538566842 398897152 34771 33554432000 4194304 5683644 140736575542208 18446744073709551615 46988129520594 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25917/statm: 97387 34771 1826 364 0 33311 0
Current children cumulated CPU time (s) 2.87
Current children cumulated vsize (KiB) 389548

[startup+6.30083 s]
/proc/loadavg: 5.70 4.24 4.23 6/199 25922
/proc/meminfo: memFree=19663048/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=389548 CPUtime=6.07 cores=1,3,5,7
/proc/25917/stat : 25917 (python) R 25915 25917 25339 0 -1 4202496 49559 935 1 0 591 16 0 0 25 0 1 0 538566842 398897152 34771 33554432000 4194304 5683644 140736575542208 18446744073709551615 5003632 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25917/statm: 97387 34771 1826 364 0 33311 0
Current children cumulated CPU time (s) 6.07
Current children cumulated vsize (KiB) 389548

[startup+12.7007 s]
/proc/loadavg: 5.59 4.26 4.24 6/199 25922
/proc/meminfo: memFree=19580944/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=389548 CPUtime=12.47 cores=1,3,5,7
/proc/25917/stat : 25917 (python) R 25915 25917 25339 0 -1 4202496 49560 935 1 0 1231 16 0 0 25 0 1 0 538566842 398897152 34772 33554432000 4194304 5683644 140736575542208 18446744073709551615 237282288309 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25917/statm: 97387 34772 1827 364 0 33311 0
Current children cumulated CPU time (s) 12.47
Current children cumulated vsize (KiB) 389548

[startup+25.5004 s]
/proc/loadavg: 5.50 4.29 4.25 6/199 25923
/proc/meminfo: memFree=19563860/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=389548 CPUtime=25.27 cores=1,3,5,7
/proc/25917/stat : 25917 (python) R 25915 25917 25339 0 -1 4202496 49566 935 1 0 2511 16 0 0 25 0 1 0 538566842 398897152 34778 33554432000 4194304 5683644 140736575542208 18446744073709551615 46988129376396 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25917/statm: 97387 34778 1833 364 0 33311 0
Current children cumulated CPU time (s) 25.27
Current children cumulated vsize (KiB) 389548

[startup+51.1009 s]
/proc/loadavg: 5.33 4.34 4.27 6/199 25923
/proc/meminfo: memFree=19538428/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=389548 CPUtime=50.87 cores=1,3,5,7
/proc/25917/stat : 25917 (python) R 25915 25917 25339 0 -1 4202496 49566 935 1 0 5071 16 0 0 25 0 1 0 538566842 398897152 34778 33554432000 4194304 5683644 140736575542208 18446744073709551615 46988129376396 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25917/statm: 97387 34778 1833 364 0 33311 0
Current children cumulated CPU time (s) 50.87
Current children cumulated vsize (KiB) 389548

[startup+102.308 s]
/proc/loadavg: 6.89 4.93 4.47 9/212 25938
/proc/meminfo: memFree=18849708/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=422208 CPUtime=55.76 cores=1,3,5,7
/proc/25917/stat : 25917 (python) S 25915 25917 25339 0 -1 4202496 52287 935 1 0 5559 17 0 0 25 0 5 0 538566842 432340992 32748 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/25917/statm: 105552 32748 1881 364 0 41461 0
[pid=25924] ppid=25917 vsize=381196 CPUtime=0.06 cores=1,3,5,7
/proc/25924/stat : 25924 (python) S 25917 25917 25339 0 -1 4202560 997 0 0 0 3 3 0 0 18 0 1 0 538572434 390344704 31233 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899727 0 0 16781312 514 4879966855604699398 0 0 17 5 0 0 0
/proc/25924/statm: 95299 31233 382 364 0 31217 0
[pid=25926] ppid=25924 vsize=66152 CPUtime=46.37 cores=1,3,5,7
/proc/25926/stat : 25926 (lingeling) R 25924 25926 25926 0 -1 4202496 33266 0 0 0 4633 4 0 0 25 0 1 0 538572435 67739648 15708 33554432000 4194304 4352236 140735992482752 18446744073709551615 4328026 0 0 16781312 17506 0 0 0 17 1 0 0 0
/proc/25926/statm: 16538 15708 129 39 0 15618 0
[pid=25927] ppid=25917 vsize=391452 CPUtime=0.06 cores=1,3,5,7
/proc/25927/stat : 25927 (python) S 25917 25917 25339 0 -1 4202560 925 0 0 0 3 3 0 0 15 0 1 0 538572437 400846848 31225 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 5630774971996274952 0 0 17 5 0 0 0
/proc/25927/statm: 97863 31225 370 364 0 33778 0
[pid=25929] ppid=25927 vsize=548696 CPUtime=46.34 cores=1,3,5,7
/proc/25929/stat : 25929 (cryptominisat-2) R 25927 25929 25929 0 -1 4202496 138976 0 0 0 4612 22 0 0 25 0 1 0 538572437 561864704 124288 33554432000 4194304 4677096 140735215324656 18446744073709551615 4249122 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25929/statm: 137174 124288 393 118 0 132133 0
[pid=25930] ppid=25917 vsize=401708 CPUtime=0.05 cores=1,3,5,7
/proc/25930/stat : 25930 (python) S 25917 25917 25339 0 -1 4202560 947 0 0 0 2 3 0 0 18 0 1 0 538572439 411348992 31229 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 4879966855604699400 0 0 17 5 0 0 0
/proc/25930/statm: 100427 31229 370 364 0 36339 0
[pid=25932] ppid=25930 vsize=66072 CPUtime=46.01 cores=1,3,5,7
/proc/25932/stat : 25932 (lingeling) R 25930 25932 25932 0 -1 4202496 33253 0 0 0 4597 4 0 0 25 0 1 0 538572440 67657728 15688 33554432000 4194304 4352236 140736929596352 18446744073709551615 4222551 0 0 16781312 17506 0 0 0 17 7 0 0 0
/proc/25932/statm: 16518 15688 129 39 0 15598 0
[pid=25933] ppid=25917 vsize=411964 CPUtime=0.06 cores=1,3,5,7
/proc/25933/stat : 25933 (python) S 25917 25917 25339 0 -1 4202560 945 0 0 0 3 3 0 0 18 0 1 0 538572442 421851136 31233 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 4879966855604699400 0 0 17 5 0 0 0
/proc/25933/statm: 102991 31233 370 364 0 38900 0
[pid=25935] ppid=25933 vsize=39020 CPUtime=46.02 cores=1,3,5,7
/proc/25935/stat : 25935 (SApperloT-base) R 25933 25935 25935 0 -1 4202496 9378 0 0 0 4600 2 0 0 25 0 1 0 538572443 39956480 8891 33554432000 134512640 135674940 4288872704 18446744073709551615 134576331 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25935/statm: 9755 8891 100 284 0 9469 0
[pid=25917/tid=25925] ppid=25915 vsize=422208 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25925/stat : 25925 (python) S 25915 25917 25339 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 5 0 538572435 432340992 32748 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 1 0 0 0
[pid=25917/tid=25928] ppid=25915 vsize=422208 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25928/stat : 25928 (python) S 25915 25917 25339 0 -1 4202560 22 935 0 0 0 0 0 0 25 0 5 0 538572437 432340992 32748 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 3 0 0 0
[pid=25917/tid=25931] ppid=25915 vsize=422208 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25931/stat : 25931 (python) S 25915 25917 25339 0 -1 4202560 27 935 0 0 0 0 0 0 25 0 5 0 538572440 432340992 32748 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=25934] ppid=25915 vsize=422208 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25934/stat : 25934 (python) S 25915 25917 25339 0 -1 4202560 30 935 0 0 0 0 0 0 25 0 5 0 538572443 432340992 32748 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 240.73
Current children cumulated vsize (KiB) 2728468

[startup+162.301 s]
/proc/loadavg: 7.04 5.33 4.64 9/216 25988
/proc/meminfo: memFree=18566180/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=432464 CPUtime=55.82 cores=1,3,5,7
/proc/25917/stat : 25917 (python) S 25915 25917 25339 0 -1 4202496 53118 935 1 0 5564 18 0 0 19 0 6 0 538566842 442843136 32755 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/25917/statm: 108116 32755 1884 364 0 44022 0
[pid=25924] ppid=25917 vsize=391440 CPUtime=0.14 cores=1,3,5,7
/proc/25924/stat : 25924 (python) S 25917 25917 25339 0 -1 4202560 1175 0 0 0 8 6 0 0 18 0 2 0 538572434 400834560 31272 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25924/statm: 97860 31272 416 364 0 33778 0
[pid=25926] ppid=25924 vsize=66148 CPUtime=106.31 cores=1,3,5,7
/proc/25926/stat : 25926 (lingeling) R 25924 25926 25926 0 -1 4202496 33266 0 0 0 10627 4 0 0 25 0 1 0 538572435 67735552 15707 33554432000 4194304 4352236 140735992482752 18446744073709551615 4328114 0 0 16781312 17506 0 0 0 17 1 0 0 0
/proc/25926/statm: 16537 15707 129 39 0 15617 0
[pid=25924/tid=25984] ppid=25917 vsize=391440 CPUtime=0 cores=1,3,5,7
/proc/25924/task/25984/stat : 25984 (python) S 25917 25917 25339 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 538582447 400834560 31272 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604471152374912 0 0 -1 5 0 0 0
[pid=25927] ppid=25917 vsize=391452 CPUtime=0.17 cores=1,3,5,7
/proc/25927/stat : 25927 (python) S 25917 25917 25339 0 -1 4202560 950 0 0 0 9 8 0 0 15 0 1 0 538572437 400846848 31225 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 4879966855604699400 0 0 17 5 0 0 0
/proc/25927/statm: 97863 31225 370 364 0 33778 0
[pid=25929] ppid=25927 vsize=576688 CPUtime=106.33 cores=1,3,5,7
/proc/25929/stat : 25929 (cryptominisat-2) R 25927 25929 25929 0 -1 4202496 146095 0 0 0 10609 24 0 0 25 0 1 0 538572437 590528512 131407 33554432000 4194304 4677096 140735215324656 18446744073709551615 4319754 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25929/statm: 144172 131407 393 118 0 139131 0
[pid=25930] ppid=25917 vsize=401708 CPUtime=0.13 cores=1,3,5,7
/proc/25930/stat : 25930 (python) S 25917 25917 25339 0 -1 4202560 966 0 0 0 6 7 0 0 18 0 1 0 538572439 411348992 31229 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 4879966855604699400 0 0 17 5 0 0 0
/proc/25930/statm: 100427 31229 370 364 0 36339 0
[pid=25932] ppid=25930 vsize=66068 CPUtime=105.65 cores=1,3,5,7
/proc/25932/stat : 25932 (lingeling) R 25930 25932 25932 0 -1 4202496 33253 0 0 0 10561 4 0 0 25 0 1 0 538572440 67653632 15687 33554432000 4194304 4352236 140736929596352 18446744073709551615 4329086 0 0 16781312 17506 0 0 0 17 7 0 0 0
/proc/25932/statm: 16517 15687 129 39 0 15597 0
[pid=25933] ppid=25917 vsize=411964 CPUtime=0.13 cores=1,3,5,7
/proc/25933/stat : 25933 (python) S 25917 25917 25339 0 -1 4202560 1113 0 0 0 7 6 0 0 19 0 2 0 538572442 421851136 31263 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/25933/statm: 102991 31263 399 364 0 38900 0
[pid=25935] ppid=25933 vsize=52200 CPUtime=99.58 cores=1,3,5,7
/proc/25935/stat : 25935 (SApperloT-base) T 25933 25935 25935 0 -1 4202496 12612 0 0 0 9954 4 0 0 25 0 1 0 538572443 53452800 12125 33554432000 134512640 135674940 4288872704 18446744073709551615 134564250 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/25935/statm: 13050 12125 100 284 0 12764 0
[pid=25933/tid=25985] ppid=25917 vsize=411964 CPUtime=0 cores=1,3,5,7
/proc/25933/task/25985/stat : 25985 (python) S 25917 25917 25339 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 538582457 421851136 31263 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604459777992896 0 0 -1 5 0 0 0
[pid=25986] ppid=25917 vsize=422220 CPUtime=0.01 cores=1,3,5,7
/proc/25986/stat : 25986 (python) S 25917 25917 25339 0 -1 4202560 931 0 0 0 0 1 0 0 15 0 1 0 538582459 432353280 31237 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 10397760960533070088 0 0 17 5 0 0 0
/proc/25986/statm: 105555 31237 370 364 0 41461 0
[pid=25988] ppid=25986 vsize=343456 CPUtime=6.07 cores=1,3,5,7
/proc/25988/stat : 25988 (cryptominisat-2) R 25986 25988 25988 0 -1 4202496 80879 0 0 0 593 14 0 0 25 0 1 0 538582460 351698944 76185 33554432000 4194304 4677096 140735382750480 18446744073709551615 4319384 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/25988/statm: 85864 76185 362 118 0 80823 0
[pid=25917/tid=25925] ppid=25915 vsize=432464 CPUtime=0 cores=1,3,5,7

################
# More data... #
################

/proc/25986/statm: 105555 31268 400 364 0 41461 0
[pid=25988] ppid=25986 vsize=616644 CPUtime=681.7 cores=1,3,5,7
/proc/25988/stat : 25988 (cryptominisat-2) T 25986 25988 25988 0 -1 4202496 239505 0 0 0 68109 61 0 0 25 0 1 0 538582460 631443456 141187 33554432000 4194304 4677096 140735382750480 18446744073709551615 4320006 0 0 16781312 2 18446744071562691360 0 0 17 3 0 0 0
/proc/25988/statm: 154161 141187 393 118 0 149120 0
[pid=25986/tid=26063] ppid=25917 vsize=422220 CPUtime=0 cores=1,3,5,7
/proc/25986/task/26063/stat : 26063 (python) S 25917 25917 25339 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 538650791 432353280 31268 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604459628577920 0 0 -1 5 0 0 0
[pid=26051] ppid=25917 vsize=432476 CPUtime=0.74 cores=1,3,5,7
/proc/26051/stat : 26051 (python) S 25917 25917 25339 0 -1 4202560 1158 0 0 0 38 36 0 0 18 0 2 0 538631077 442855424 31271 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/26051/statm: 108119 31271 399 364 0 44022 0
[pid=26053] ppid=26051 vsize=183160 CPUtime=503.81 cores=1,3,5,7
/proc/26053/stat : 26053 (precosat) R 26051 26053 26053 0 -1 4202496 46335 0 0 0 50374 7 0 0 25 0 1 0 538631077 187555840 43202 33554432000 4194304 4890832 140736063956304 18446744073709551615 4238599 0 0 16781312 17442 0 0 0 17 1 0 0 0
/proc/26053/statm: 45790 43202 86 171 0 45617 0
[pid=26051/tid=26076] ppid=25917 vsize=432476 CPUtime=0 cores=1,3,5,7
/proc/26051/task/26076/stat : 26076 (python) S 25917 25917 25339 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 538670229 442855424 31271 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604471121520512 0 0 -1 5 0 0 0
[pid=26064] ppid=25917 vsize=442732 CPUtime=0.17 cores=1,3,5,7
/proc/26064/stat : 26064 (python) S 25917 25917 25339 0 -1 4202560 1135 0 0 0 9 8 0 0 16 0 2 0 538650794 453357568 31275 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/26064/statm: 110683 31275 399 364 0 46583 0
[pid=26066] ppid=26064 vsize=573316 CPUtime=102.36 cores=1,3,5,7
/proc/26066/stat : 26066 (cryptominisat-2) T 26064 26066 26066 0 -1 4202496 145450 0 0 0 10213 23 0 0 25 0 1 0 538650794 587075584 130743 33554432000 4194304 4677096 140735232209488 18446744073709551615 4320006 0 0 16781312 2 18446744071562691360 0 0 17 1 0 0 0
/proc/26066/statm: 143329 130743 393 118 0 138288 0
[pid=26064/tid=26070] ppid=25917 vsize=442732 CPUtime=0 cores=1,3,5,7
/proc/26064/task/26070/stat : 26070 (python) S 25917 25917 25339 0 -1 4202560 49 0 0 0 0 0 0 0 16 0 2 0 538661078 453357568 31275 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604459786076288 0 0 -1 5 0 0 0
[pid=26071] ppid=25917 vsize=452988 CPUtime=0.45 cores=1,3,5,7
/proc/26071/stat : 26071 (python) S 25917 25917 25339 0 -1 4202560 1196 0 0 0 22 23 0 0 19 0 2 0 538661080 463859712 31285 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/26071/statm: 113247 31285 405 364 0 49144 0
[pid=26073] ppid=26071 vsize=66820 CPUtime=296.8 cores=1,3,5,7
/proc/26073/stat : 26073 (lingeling) T 26071 26073 26073 0 -1 4202496 41638 0 0 0 29674 6 0 0 25 0 1 0 538661081 68423680 15875 33554432000 4194304 4352236 140733579687872 18446744073709551615 4328026 0 0 16781312 17506 18446744071562691360 0 0 17 1 0 0 0
/proc/26073/statm: 16705 15875 129 39 0 15785 0
[pid=26071/tid=26088] ppid=25917 vsize=452988 CPUtime=0 cores=1,3,5,7
/proc/26071/task/26088/stat : 26088 (python) S 25917 25917 25339 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538690830 463859712 31285 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604471125407936 0 0 -1 1 0 0 0
[pid=26089] ppid=25917 vsize=463244 CPUtime=0 cores=1,3,5,7
/proc/26089/stat : 26089 (python) S 25917 25917 25339 0 -1 4202560 943 0 0 0 0 0 0 0 18 0 1 0 538690833 474361856 31253 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 4419337458559221759 0 0 17 5 0 0 0
/proc/26089/statm: 115811 31253 370 364 0 51705 0
[pid=26091] ppid=26089 vsize=175084 CPUtime=4.77 cores=1,3,5,7
/proc/26091/stat : 26091 (precosat) R 26089 26091 26091 0 -1 4202496 44314 0 0 0 471 6 0 0 25 0 1 0 538690833 179286016 41177 33554432000 4194304 4890832 140736819573328 18446744073709551615 4238673 0 0 16781312 17442 0 0 0 17 5 0 0 0
/proc/26091/statm: 43771 41177 86 171 0 43598 0
[pid=25917/tid=25925] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25925/stat : 25925 (python) S 25915 25917 25339 0 -1 4202560 38 935 0 0 0 0 0 0 18 0 10 0 538572435 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 1 0 0 0
[pid=25917/tid=25928] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25928/stat : 25928 (python) S 25915 25917 25339 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 10 0 538572437 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 3 0 0 0
[pid=25917/tid=25931] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25931/stat : 25931 (python) S 25915 25917 25339 0 -1 4202560 27 935 0 0 0 0 0 0 25 0 10 0 538572440 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=25934] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25934/stat : 25934 (python) S 25915 25917 25339 0 -1 4202560 30 935 0 0 0 0 0 0 25 0 10 0 538572443 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=25987] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25987/stat : 25987 (python) S 25915 25917 25339 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 10 0 538582460 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=26052] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/26052/stat : 26052 (python) S 25915 25917 25339 0 -1 4202560 31 935 0 0 0 0 0 0 18 0 10 0 538631077 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=26065] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/26065/stat : 26065 (python) S 25915 25917 25339 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 10 0 538650794 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=26072] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/26072/stat : 26072 (python) S 25915 25917 25339 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 10 0 538661080 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=26090] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/26090/stat : 26090 (python) S 25915 25917 25339 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 10 0 538690833 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.15
Current children cumulated vsize (KiB) 6737932

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1244.7 s]
/proc/loadavg: 8.19 8.07 7.01 10/242 26091
/proc/meminfo: memFree=17444808/32950904 swapFree=67082284/67111528
[pid=25917] ppid=25915 vsize=473488 CPUtime=56.04 cores=1,3,5,7
/proc/25917/stat : 25917 (python) S 25915 25917 25339 0 -1 4202496 56503 935 1 0 5584 20 0 0 20 0 10 0 538566842 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/25917/statm: 118372 32779 1892 364 0 54266 0
[pid=25924] ppid=25917 vsize=391440 CPUtime=1.48 cores=1,3,5,7
/proc/25924/stat : 25924 (python) S 25917 25917 25339 0 -1 4202560 1204 0 0 0 79 69 0 0 18 0 2 0 538572434 400834560 31272 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25924/statm: 97860 31272 416 364 0 33778 0
[pid=25926] ppid=25924 vsize=87340 CPUtime=1070.91 cores=1,3,5,7
/proc/25926/stat : 25926 (lingeling) T 25924 25926 25926 0 -1 4202496 48807 0 0 0 107085 6 0 0 25 0 1 0 538572435 89436160 21005 33554432000 4194304 4352236 140735992482752 18446744073709551615 4328390 0 0 16781312 17506 18446744071562691360 0 0 17 1 0 0 0
/proc/25926/statm: 21835 21005 129 39 0 20915 0
[pid=25924/tid=25984] ppid=25917 vsize=391440 CPUtime=0 cores=1,3,5,7
/proc/25924/task/25984/stat : 25984 (python) S 25917 25917 25339 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 538582447 400834560 31272 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604471152374912 0 0 -1 5 0 0 0
[pid=25927] ppid=25917 vsize=391452 CPUtime=1.21 cores=1,3,5,7
/proc/25927/stat : 25927 (python) S 25917 25917 25339 0 -1 4202560 1170 0 0 0 61 60 0 0 15 0 2 0 538572437 400846848 31256 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/25927/statm: 97863 31256 400 364 0 33778 0
[pid=25929] ppid=25927 vsize=616492 CPUtime=799.08 cores=1,3,5,7
/proc/25929/stat : 25929 (cryptominisat-2) R 25927 25929 25929 0 -1 4202496 249092 0 0 0 79845 63 0 0 25 0 1 0 538572437 631287808 142227 33554432000 4194304 4677096 140735215324656 18446744073709551615 4319592 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/25929/statm: 154123 142227 393 118 0 149082 0
[pid=25927/tid=26050] ppid=25917 vsize=391452 CPUtime=0 cores=1,3,5,7
/proc/25927/task/26050/stat : 26050 (python) S 25917 25917 25339 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 538631074 400846848 31256 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604466862041792 0 0 -1 3 0 0 0
[pid=25930] ppid=25917 vsize=401708 CPUtime=1.65 cores=1,3,5,7
/proc/25930/stat : 25930 (python) S 25917 25917 25339 0 -1 4202560 996 0 0 0 89 76 0 0 18 0 1 0 538572439 411348992 31229 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 4879966855604699400 0 0 17 5 0 0 0
/proc/25930/statm: 100427 31229 370 364 0 36339 0
[pid=25932] ppid=25930 vsize=83164 CPUtime=1178.24 cores=1,3,5,7
/proc/25932/stat : 25932 (lingeling) R 25930 25932 25932 0 -1 4202496 51866 0 0 0 117817 7 0 0 25 0 1 0 538572440 85159936 19961 33554432000 4194304 4352236 140736929596352 18446744073709551615 4328100 0 0 16781312 17506 0 0 0 17 7 0 0 0
/proc/25932/statm: 20791 19961 129 39 0 19871 0
[pid=25933] ppid=25917 vsize=411964 CPUtime=0.13 cores=1,3,5,7
/proc/25933/stat : 25933 (python) S 25917 25917 25339 0 -1 4202560 1113 0 0 0 7 6 0 0 19 0 2 0 538572442 421851136 31263 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/25933/statm: 102991 31263 399 364 0 38900 0
[pid=25935] ppid=25933 vsize=52200 CPUtime=99.58 cores=1,3,5,7
/proc/25935/stat : 25935 (SApperloT-base) T 25933 25935 25935 0 -1 4202496 12612 0 0 0 9954 4 0 0 25 0 1 0 538572443 53452800 12125 33554432000 134512640 135674940 4288872704 18446744073709551615 134564250 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/25935/statm: 13050 12125 100 284 0 12764 0
[pid=25933/tid=25985] ppid=25917 vsize=411964 CPUtime=0 cores=1,3,5,7
/proc/25933/task/25985/stat : 25985 (python) S 25917 25917 25339 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 538582457 421851136 31263 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604459777992896 0 0 -1 5 0 0 0
[pid=25986] ppid=25917 vsize=422220 CPUtime=1.03 cores=1,3,5,7
/proc/25986/stat : 25986 (python) S 25917 25917 25339 0 -1 4202560 1175 0 0 0 56 47 0 0 16 0 2 0 538582459 432353280 31268 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/25986/statm: 105555 31268 400 364 0 41461 0
[pid=25988] ppid=25986 vsize=616644 CPUtime=681.7 cores=1,3,5,7
/proc/25988/stat : 25988 (cryptominisat-2) T 25986 25988 25988 0 -1 4202496 239505 0 0 0 68109 61 0 0 25 0 1 0 538582460 631443456 141187 33554432000 4194304 4677096 140735382750480 18446744073709551615 4320006 0 0 16781312 2 18446744071562691360 0 0 17 3 0 0 0
/proc/25988/statm: 154161 141187 393 118 0 149120 0
[pid=25986/tid=26063] ppid=25917 vsize=422220 CPUtime=0 cores=1,3,5,7
/proc/25986/task/26063/stat : 26063 (python) S 25917 25917 25339 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 538650791 432353280 31268 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604459628577920 0 0 -1 5 0 0 0
[pid=26051] ppid=25917 vsize=432476 CPUtime=0.74 cores=1,3,5,7
/proc/26051/stat : 26051 (python) S 25917 25917 25339 0 -1 4202560 1158 0 0 0 38 36 0 0 18 0 2 0 538631077 442855424 31271 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/26051/statm: 108119 31271 399 364 0 44022 0
[pid=26053] ppid=26051 vsize=183160 CPUtime=503.81 cores=1,3,5,7
/proc/26053/stat : 26053 (precosat) R 26051 26053 26053 0 -1 4202496 46335 0 0 0 50374 7 0 0 25 0 1 0 538631077 187555840 43202 33554432000 4194304 4890832 140736063956304 18446744073709551615 4238599 0 0 16781312 17442 0 0 0 17 1 0 0 0
/proc/26053/statm: 45790 43202 86 171 0 45617 0
[pid=26051/tid=26076] ppid=25917 vsize=432476 CPUtime=0 cores=1,3,5,7
/proc/26051/task/26076/stat : 26076 (python) S 25917 25917 25339 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 538670229 442855424 31271 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604471121520512 0 0 -1 5 0 0 0
[pid=26064] ppid=25917 vsize=442732 CPUtime=0.17 cores=1,3,5,7
/proc/26064/stat : 26064 (python) S 25917 25917 25339 0 -1 4202560 1135 0 0 0 9 8 0 0 16 0 2 0 538650794 453357568 31275 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/26064/statm: 110683 31275 399 364 0 46583 0
[pid=26066] ppid=26064 vsize=573316 CPUtime=102.36 cores=1,3,5,7
/proc/26066/stat : 26066 (cryptominisat-2) T 26064 26066 26066 0 -1 4202496 145450 0 0 0 10213 23 0 0 25 0 1 0 538650794 587075584 130743 33554432000 4194304 4677096 140735232209488 18446744073709551615 4320006 0 0 16781312 2 18446744071562691360 0 0 17 1 0 0 0
/proc/26066/statm: 143329 130743 393 118 0 138288 0
[pid=26064/tid=26070] ppid=25917 vsize=442732 CPUtime=0 cores=1,3,5,7
/proc/26064/task/26070/stat : 26070 (python) S 25917 25917 25339 0 -1 4202560 49 0 0 0 0 0 0 0 16 0 2 0 538661078 453357568 31275 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604459786076288 0 0 -1 5 0 0 0
[pid=26071] ppid=25917 vsize=452988 CPUtime=0.45 cores=1,3,5,7
/proc/26071/stat : 26071 (python) S 25917 25917 25339 0 -1 4202560 1196 0 0 0 22 23 0 0 19 0 2 0 538661080 463859712 31285 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290706379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/26071/statm: 113247 31285 405 364 0 49144 0
[pid=26073] ppid=26071 vsize=66820 CPUtime=296.8 cores=1,3,5,7
/proc/26073/stat : 26073 (lingeling) T 26071 26073 26073 0 -1 4202496 41638 0 0 0 29674 6 0 0 25 0 1 0 538661081 68423680 15875 33554432000 4194304 4352236 140733579687872 18446744073709551615 4328026 0 0 16781312 17506 18446744071562691360 0 0 17 1 0 0 0
/proc/26073/statm: 16705 15875 129 39 0 15785 0
[pid=26071/tid=26088] ppid=25917 vsize=452988 CPUtime=0 cores=1,3,5,7
/proc/26071/task/26088/stat : 26088 (python) S 25917 25917 25339 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538690830 463859712 31285 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 514 18446604471125407936 0 0 -1 1 0 0 0
[pid=26089] ppid=25917 vsize=463244 CPUtime=0 cores=1,3,5,7
/proc/26089/stat : 26089 (python) S 25917 25917 25339 0 -1 4202560 943 0 0 0 0 0 0 0 18 0 1 0 538690833 474361856 31253 33554432000 4194304 5683644 140736575542208 18446744073709551615 237278899782 0 0 16781312 514 4419337458559221759 0 0 17 5 0 0 0
/proc/26089/statm: 115811 31253 370 364 0 51705 0
[pid=26091] ppid=26089 vsize=175084 CPUtime=4.77 cores=1,3,5,7
/proc/26091/stat : 26091 (precosat) R 26089 26091 26091 0 -1 4202496 44314 0 0 0 471 6 0 0 25 0 1 0 538690833 179286016 41177 33554432000 4194304 4890832 140736819573328 18446744073709551615 4238673 0 0 16781312 17442 0 0 0 17 5 0 0 0
/proc/26091/statm: 43771 41177 86 171 0 43598 0
[pid=25917/tid=25925] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25925/stat : 25925 (python) S 25915 25917 25339 0 -1 4202560 38 935 0 0 0 0 0 0 18 0 10 0 538572435 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 1 0 0 0
[pid=25917/tid=25928] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25928/stat : 25928 (python) S 25915 25917 25339 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 10 0 538572437 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 3 0 0 0
[pid=25917/tid=25931] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25931/stat : 25931 (python) S 25915 25917 25339 0 -1 4202560 27 935 0 0 0 0 0 0 25 0 10 0 538572440 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=25934] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25934/stat : 25934 (python) S 25915 25917 25339 0 -1 4202560 30 935 0 0 0 0 0 0 25 0 10 0 538572443 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=25987] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/25987/stat : 25987 (python) S 25915 25917 25339 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 10 0 538582460 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=26052] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/26052/stat : 26052 (python) S 25915 25917 25339 0 -1 4202560 31 935 0 0 0 0 0 0 18 0 10 0 538631077 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=26065] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/26065/stat : 26065 (python) S 25915 25917 25339 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 10 0 538650794 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=26072] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/26072/stat : 26072 (python) S 25915 25917 25339 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 10 0 538661080 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
[pid=25917/tid=26090] ppid=25915 vsize=473488 CPUtime=0 cores=1,3,5,7
/proc/25917/task/26090/stat : 26090 (python) S 25915 25917 25339 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 10 0 538690833 484851712 32779 33554432000 4194304 5683644 140736575542208 18446744073709551615 237290703281 0 0 16781312 2 18446604468344308992 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.15
Current children cumulated vsize (KiB) 6737932

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25917 and gives
#  childrusage.ru_utime.tv_sec=1237
#  childrusage.ru_utime.tv_usec=695841
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=592453
# CPU time returned by wait4() is 1241.29
# while last known CPU time is 4800.15
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1244.76
CPU time (s): 4800.15
CPU user time (s): 4794.87
CPU system time (s): 5.28
CPU usage (%): 385.628
Max. virtual memory (cumulated for all children) (KiB): 6737932

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1237.7
system time used= 3.59245
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 119537
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8484
involuntary context switches= 14329

runsolver used 3.19651 second user time and 7.61884 second system time

The end

Launcher Data 

Begin job on node121 at 2011-05-01 19:07:51
IDJOB=3317914
IDBENCH=82481
IDSOLVER=1706
FILE ID=node121/3317914-1304269670
RUNJOBID= node121-1304267283-25358
PBS_JOBID= 13173958
Free space on /tmp= 73568 MiB

SOLVER NAME= borg-sat 11.03.02-industrial
BENCH NAME= SAT11/application/jarvisalo/HWMCC10-combination-timeframe-expansion/hwmcc10-timeframe-expansion-k50-pdtvisns3p00-tseitin.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317914-1304269670/watcher-3317914-1304269670 -o /tmp/evaluation-result-3317914-1304269670/solver-3317914-1304269670 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317914-1304269670.cnf -seed 824782661 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= a8ad0173824ed9404a8c153da07c5f9c
RANDOM SEED=824782661

node121.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950904 kB
MemFree:      19854688 kB
Buffers:       2193148 kB
Cached:        9527980 kB
SwapCached:       2776 kB
Active:        9520868 kB
Inactive:      3052136 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950904 kB
LowFree:      19854688 kB
SwapTotal:    67111528 kB
SwapFree:     67082284 kB
Dirty:           77296 kB
Writeback:           0 kB
AnonPages:      849416 kB
Mapped:          21580 kB
Slab:           452184 kB
PageTables:       9256 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586980 kB
Committed_AS:  1424120 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73540 MiB
End job on node121 at 2011-05-01 19:28:38