Trace number 3321581

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-industrial? (MO) 2132.65 689.63

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-two-cities-sequential-15nodes-1000size-3degree-100mindistance-3trucks-10packages-2008seed.040-SAT.cnf
MD5SUM74e0fa52e4789615985c73d0a8b105b4
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark124.843
Satisfiable
(Un)Satisfiability was proved
Number of variables580875
Number of clauses3140965
Sum of the clauses size7465165
Maximum clause length602
Minimum clause length1
Number of clauses of size 1445
Number of clauses of size 23105720
Number of clauses of size 3720
Number of clauses of size 4720
Number of clauses of size 51200
Number of clauses of size over 532160

Solver Data 

0.36/0.68	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.36/0.68	c INFO: solving HOME/instance-3321581-1304347353.cnf
0.36/0.68	c INFO: computing task features
13.13/13.48	c INFO: computed variable-clause graph statistics
13.53/13.89	c INFO: computed clause balance statistics
58.84/59.14	c INFO: computed variable balance statistics
90.14/90.48	c INFO: computed small-clause counts
91.94/92.25	c INFO: computed Horn-clause counts
481.64/481.93	c INFO: computed variable graph statistics
481.64/481.94	c INFO: collected features for HOME/instance-3321581-1304347353.cnf in 480.89 s
481.64/481.97	c INFO: running cryptominisat-2.9.0@0 for 582 with 39518 remaining (b = 0.39)
481.74/482.01	c INFO: running lingeling-276@0 for 97 with 39518 remaining (b = 0.19)
481.74/482.05	c INFO: running cryptominisat-2.9.0@0 for 1649 with 39518 remaining (b = 0.49)
481.74/482.09	c INFO: running lingeling-276@0 for 1261 with 39518 remaining (b = 0.33)
481.84/482.13	c INFO: running cryptominisat-2.9.0@0 for 97 with 39518 remaining (b = 0.02)
481.84/482.17	c INFO: running cryptominisat-2.9.0@0 for 1649 with 39518 remaining (b = 0.39)
481.94/482.21	c INFO: running glucose@0 for 5821 with 39518 remaining (b = 0.57)
481.94/482.25	c INFO: running cryptominisat-2.9.0@0 for 1940 with 39518 remaining (b = 0.34)
1281.41/582.25	c INFO: running lingeling-276@0 for 291 with 39418 remaining (b = 0.08)
1302.97/584.98	c INFO: running glucose@0 for 5821 with 39415 remaining (b = 0.56)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321581-1304347353/watcher-3321581-1304347353 -o /tmp/evaluation-result-3321581-1304347353/solver-3321581-1304347353 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3321581-1304347353.cnf -seed 1796958160 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.61 2.65 3.74 2/263 28707
/proc/meminfo: memFree=21405576/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=15388 CPUtime=0 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 817 0 0 0 0 0 0 0 17 0 1 0 624965966 15757312 702 33554432000 4194304 5683644 140737488177072 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/28707/statm: 3847 704 334 364 0 334 0

[startup+0.0523711 s]
/proc/loadavg: 4.61 2.65 3.74 2/263 28707
/proc/meminfo: memFree=21405576/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=23188 CPUtime=0.03 cores=0-7
/proc/28707/stat : 28707 (python) S 28705 28707 27951 0 -1 4202496 1324 0 0 0 3 0 0 0 17 0 1 0 624965966 23744512 1173 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458063872 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/28707/statm: 5797 1173 432 364 0 724 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 23188

[startup+0.100354 s]
/proc/loadavg: 4.61 2.65 3.74 2/263 28707
/proc/meminfo: memFree=21405576/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=85624 CPUtime=0.05 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 2631 0 0 0 4 1 0 0 17 0 1 0 624965966 87678976 2348 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458063872 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/28707/statm: 21406 2348 713 364 0 1623 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 85624

[startup+0.300303 s]
/proc/loadavg: 4.61 2.65 3.74 2/263 28707
/proc/meminfo: memFree=21405576/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=135752 CPUtime=0.19 cores=0-7
/proc/28707/stat : 28707 (python) S 28705 28707 27951 0 -1 4202496 5961 933 1 0 14 5 0 0 17 0 1 0 624965966 139010048 4150 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458061829 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/28707/statm: 33938 4150 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135752

[startup+0.700226 s]
/proc/loadavg: 4.61 2.65 3.74 2/263 28707
/proc/meminfo: memFree=21405576/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=338068 CPUtime=0.43 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 19084 933 1 0 31 12 0 0 16 0 1 0 624965966 346181632 16766 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458065664 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/28707/statm: 84517 16791 1758 364 0 20441 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 338068

[startup+1.50104 s]
/proc/loadavg: 4.61 2.65 3.74 2/264 28712
/proc/meminfo: memFree=21306244/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=391516 CPUtime=1.24 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 37054 933 1 0 109 15 0 0 17 0 1 0 624965966 400912384 34737 33554432000 4194304 5683644 140737488177072 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/28707/statm: 97879 34738 1761 364 0 33803 0
Current children cumulated CPU time (s) 1.24
Current children cumulated vsize (KiB) 391516

[startup+3.10074 s]
/proc/loadavg: 4.32 2.62 3.73 2/264 28712
/proc/meminfo: memFree=21231596/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=496332 CPUtime=2.84 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 62866 933 1 0 265 19 0 0 19 0 1 0 624965966 508243968 60548 33554432000 4194304 5683644 140737488177072 18446744073709551615 4273416 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/28707/statm: 124083 60549 1761 364 0 60007 0
Current children cumulated CPU time (s) 2.84
Current children cumulated vsize (KiB) 496332

[startup+6.30016 s]
/proc/loadavg: 4.32 2.62 3.73 2/264 28712
/proc/meminfo: memFree=21023284/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=706656 CPUtime=6.03 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 114290 933 1 0 577 26 0 0 25 0 1 0 624965966 723615744 111972 33554432000 4194304 5683644 140737488177072 18446744073709551615 4602336 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/28707/statm: 176664 111972 1761 364 0 112588 0
Current children cumulated CPU time (s) 6.03
Current children cumulated vsize (KiB) 706656

[startup+12.701 s]
/proc/loadavg: 3.81 2.57 3.70 2/264 28712
/proc/meminfo: memFree=20841968/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=821580 CPUtime=12.43 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 254695 933 1 0 1199 44 0 0 25 0 1 0 624965966 841297920 142800 33554432000 4194304 5683644 140737488177072 18446744073709551615 47445208610368 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/28707/statm: 205395 142800 1815 364 0 141319 0
Current children cumulated CPU time (s) 12.43
Current children cumulated vsize (KiB) 821580

[startup+25.5006 s]
/proc/loadavg: 3.38 2.52 3.67 2/264 28713
/proc/meminfo: memFree=20867780/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=797040 CPUtime=25.24 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 266944 933 1 0 2478 46 0 0 25 0 1 0 624965966 816168960 136676 33554432000 4194304 5683644 140737488177072 18446744073709551615 47445208446607 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/28707/statm: 199260 136676 1826 364 0 135184 0
Current children cumulated CPU time (s) 25.24
Current children cumulated vsize (KiB) 797040

[startup+51.1008 s]
/proc/loadavg: 2.57 2.39 3.60 2/264 28713
/proc/meminfo: memFree=20867764/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=797040 CPUtime=50.84 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 266944 933 1 0 5038 46 0 0 25 0 1 0 624965966 816168960 136676 33554432000 4194304 5683644 140737488177072 18446744073709551615 4567008 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/28707/statm: 199260 136676 1826 364 0 135184 0
Current children cumulated CPU time (s) 50.84
Current children cumulated vsize (KiB) 797040

[startup+102.309 s]
/proc/loadavg: 1.62 2.16 3.44 2/264 28715
/proc/meminfo: memFree=20842780/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=821580 CPUtime=102.04 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 291458 933 1 0 10156 48 0 0 25 0 1 0 624965966 841297920 142818 33554432000 4194304 5683644 140737488177072 18446744073709551615 47445208197260 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/28707/statm: 205395 142818 1833 364 0 141319 0
Current children cumulated CPU time (s) 102.04
Current children cumulated vsize (KiB) 821580

[startup+162.3 s]
/proc/loadavg: 1.23 1.94 3.29 2/264 28716
/proc/meminfo: memFree=20842856/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=821580 CPUtime=162.03 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 291458 933 1 0 16155 48 0 0 25 0 1 0 624965966 841297920 142818 33554432000 4194304 5683644 140737488177072 18446744073709551615 47445208197260 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/28707/statm: 205395 142818 1833 364 0 141319 0
Current children cumulated CPU time (s) 162.03
Current children cumulated vsize (KiB) 821580

[startup+222.301 s]
/proc/loadavg: 1.14 1.78 3.15 2/264 28718
/proc/meminfo: memFree=20843072/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=821580 CPUtime=222.04 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 291458 933 1 0 22156 48 0 0 25 0 1 0 624965966 841297920 142818 33554432000 4194304 5683644 140737488177072 18446744073709551615 47445208197260 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/28707/statm: 205395 142818 1833 364 0 141319 0
Current children cumulated CPU time (s) 222.04
Current children cumulated vsize (KiB) 821580

[startup+282.301 s]
/proc/loadavg: 1.05 1.64 3.01 2/264 28720
/proc/meminfo: memFree=20843292/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=821580 CPUtime=282.04 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 291458 933 1 0 28156 48 0 0 25 0 1 0 624965966 841297920 142818 33554432000 4194304 5683644 140737488177072 18446744073709551615 47445208197260 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/28707/statm: 205395 142818 1833 364 0 141319 0
Current children cumulated CPU time (s) 282.04
Current children cumulated vsize (KiB) 821580

[startup+342.301 s]
/proc/loadavg: 1.02 1.52 2.88 2/264 28721
/proc/meminfo: memFree=20843380/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=821580 CPUtime=342.04 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 291458 933 1 0 34156 48 0 0 25 0 1 0 624965966 841297920 142818 33554432000 4194304 5683644 140737488177072 18446744073709551615 47445208197260 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/28707/statm: 205395 142818 1833 364 0 141319 0
Current children cumulated CPU time (s) 342.04
Current children cumulated vsize (KiB) 821580

[startup+402.301 s]
/proc/loadavg: 1.00 1.42 2.76 2/264 28723
/proc/meminfo: memFree=20843700/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=821580 CPUtime=402.04 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 291458 933 1 0 40156 48 0 0 25 0 1 0 624965966 841297920 142818 33554432000 4194304 5683644 140737488177072 18446744073709551615 47445208197248 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/28707/statm: 205395 142818 1833 364 0 141319 0
Current children cumulated CPU time (s) 402.04
Current children cumulated vsize (KiB) 821580

[startup+462.3 s]
/proc/loadavg: 1.00 1.34 2.65 2/264 28724
/proc/meminfo: memFree=20843668/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=821580 CPUtime=462.04 cores=0-7
/proc/28707/stat : 28707 (python) R 28705 28707 27951 0 -1 4202496 291458 933 1 0 46156 48 0 0 25 0 1 0 624965966 841297920 142818 33554432000 4194304 5683644 140737488177072 18446744073709551615 47445208197260 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/28707/statm: 205395 142818 1833 364 0 141319 0
Current children cumulated CPU time (s) 462.04
Current children cumulated vsize (KiB) 821580

[startup+522.301 s]
/proc/loadavg: 4.53 2.19 2.85 9/288 28750
/proc/meminfo: memFree=17062664/32951124 swapFree=67111348/67111528
[pid=28707] ppid=28705 vsize=679936 CPUtime=482 cores=0-7
/proc/28707/stat : 28707 (python) S 28705 28707 27951 0 -1 4202496 297348 933 1 0 48142 58 0 0 22 0 9 0 624965966 696254464 86980 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469892555 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/28707/statm: 169984 86980 1889 364 0 105881 0
[pid=28726] ppid=28707 vsize=597900 CPUtime=0.09 cores=0-7
/proc/28726/stat : 28726 (python) S 28707 28707 27951 0 -1 4202560 940 0 0 0 5 4 0 0 15 0 1 0 625014164 612249600 85435 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085903 0 0 16781312 514 12663682347514691844 0 0 17 1 0 0 0
/proc/28726/statm: 149475 85435 376 364 0 85393 0
[pid=28728] ppid=28726 vsize=968808 CPUtime=40.29 cores=0-7
/proc/28728/stat : 28728 (cryptominisat-2) R 28726 28728 28728 0 -1 4202496 206221 0 0 0 4003 26 0 0 25 0 1 0 625014165 992059392 177132 33554432000 4194304 4677096 140735960247520 18446744073709551615 4335955 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/28728/statm: 242202 177132 387 118 0 237161 0
[pid=28729] ppid=28707 vsize=608156 CPUtime=0.08 cores=0-7
/proc/28729/stat : 28729 (python) S 28707 28707 27951 0 -1 4202560 929 0 0 0 4 4 0 0 18 0 1 0 625014168 622751744 85433 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 12663682347514691844 0 0 17 2 0 0 0
/proc/28729/statm: 152039 85433 370 364 0 87954 0
[pid=28731] ppid=28729 vsize=141376 CPUtime=40.26 cores=0-7
/proc/28731/stat : 28731 (lingeling) R 28729 28731 28731 0 -1 4202496 66199 0 0 0 4015 11 0 0 25 0 1 0 625014169 144769024 34514 33554432000 4194304 4352236 140735691717632 18446744073709551615 4281237 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/28731/statm: 35344 34514 129 39 0 34424 0
[pid=28732] ppid=28707 vsize=618412 CPUtime=0.08 cores=0-7

################
# More data... #
################

[pid=28729/tid=28753] ppid=28707 vsize=608156 CPUtime=0 cores=0-7
/proc/28729/task/28753/stat : 28753 (python) S 28707 28707 27951 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 625024189 622751744 85428 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 514 18446604453848562752 0 0 -1 5 0 0 0
[pid=28732] ppid=28707 vsize=618412 CPUtime=0.62 cores=0-7
/proc/28732/stat : 28732 (python) S 28707 28707 27951 0 -1 4202560 1239 0 0 0 38 24 0 0 15 0 1 0 625014171 633253888 85407 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 1163186549458698500 0 0 17 2 0 0 0
/proc/28732/statm: 154603 85407 340 364 0 90515 0
[pid=28734] ppid=28732 vsize=4363860 CPUtime=206.74 cores=0-7
/proc/28734/stat : 28734 (cryptominisat-2) R 28732 28734 28734 0 -1 4202496 7721095 0 0 0 19156 1518 0 0 25 0 1 0 625014172 4468592640 1023347 33554432000 4194304 4677096 140736821785488 18446744073709551615 4366630 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/28734/statm: 1090965 1023348 390 118 0 1085924 0
[pid=28735] ppid=28707 vsize=628668 CPUtime=0.41 cores=0-7
/proc/28735/stat : 28735 (python) S 28707 28707 27951 0 -1 4202560 1001 0 0 0 21 20 0 0 18 0 1 0 625014175 643756032 85410 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 12663682347514691844 0 0 17 6 0 0 0
/proc/28735/statm: 157167 85410 339 364 0 93076 0
[pid=28737] ppid=28735 vsize=162284 CPUtime=203.82 cores=0-7
/proc/28737/stat : 28737 (lingeling) R 28735 28737 28737 0 -1 4202496 90046 0 0 0 20366 16 0 0 25 0 1 0 625014176 166178816 39742 33554432000 4194304 4352236 140734584733456 18446744073709551615 4319064 0 0 16781312 17506 0 0 0 17 3 0 0 0
/proc/28737/statm: 40571 39742 129 39 0 39651 0
[pid=28738] ppid=28707 vsize=638924 CPUtime=0.21 cores=0-7
/proc/28738/stat : 28738 (python) S 28707 28707 27951 0 -1 4202560 1143 0 0 0 10 11 0 0 17 0 2 0 625014179 654258176 85440 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/28738/statm: 159731 85440 364 364 0 95637 0
[pid=28740] ppid=28738 vsize=994136 CPUtime=102.51 cores=0-7
/proc/28740/stat : 28740 (cryptominisat-2) T 28738 28740 28740 0 -1 4202496 224422 0 0 0 10215 36 0 0 25 0 1 0 625014180 1017995264 182158 33554432000 4194304 4677096 140736653875648 18446744073709551615 4336125 0 0 16781312 2 18446744071562691360 0 0 17 1 0 0 0
/proc/28740/statm: 248534 182158 387 118 0 243493 0
[pid=28738/tid=28754] ppid=28707 vsize=638924 CPUtime=0 cores=0-7
/proc/28738/task/28754/stat : 28754 (python) S 28707 28707 27951 0 -1 4202560 50 0 0 0 0 0 0 0 17 0 2 0 625024461 654258176 85440 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 514 18446604453870898048 0 0 -1 1 0 0 0
[pid=28741] ppid=28707 vsize=649180 CPUtime=0.66 cores=0-7
/proc/28741/stat : 28741 (python) S 28707 28707 27951 0 -1 4202560 1328 0 0 0 39 27 0 0 15 0 1 0 625014183 664760320 85419 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 7925147875909009668 0 0 17 4 0 0 0
/proc/28741/statm: 162295 85419 340 364 0 98198 0
[pid=28743] ppid=28741 vsize=5808248 CPUtime=206.47 cores=0-7
/proc/28743/stat : 28743 (cryptominisat-2) R 28741 28743 28743 0 -1 4202496 12641382 0 0 0 18403 2244 0 0 25 0 1 0 625014184 5947645952 1387008 33554432000 4194304 4677096 140736741271920 18446744073709551615 4364705 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/28743/statm: 1452062 1387010 390 118 0 1447021 0
[pid=28744] ppid=28707 vsize=659436 CPUtime=0.42 cores=0-7
/proc/28744/stat : 28744 (python) S 28707 28707 27951 0 -1 4202560 1006 0 0 0 23 19 0 0 18 0 1 0 625014187 675262464 85422 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 12663682347514691842 0 0 17 6 0 0 0
/proc/28744/statm: 164859 85422 339 364 0 100759 0
[pid=28746] ppid=28744 vsize=459304 CPUtime=205.64 cores=0-7
/proc/28746/stat : 28746 (glucose_static) R 28744 28746 28746 0 -1 4202496 110629 0 0 0 20535 29 0 0 25 0 1 0 625014188 470327296 110307 33554432000 134512640 135217611 4289856192 18446744073709551615 134528127 0 0 16781312 8388611 0 0 0 17 4 0 0 0
/proc/28746/statm: 114826 110307 78 173 0 114651 0
[pid=28747] ppid=28707 vsize=669692 CPUtime=0.87 cores=0-7
/proc/28747/stat : 28747 (python) S 28707 28707 27951 0 -1 4202560 1336 0 0 0 56 31 0 0 15 0 1 0 625014191 685764608 85432 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 8441254235940552962 0 0 17 7 0 0 0
/proc/28747/statm: 167423 85432 345 364 0 103320 0
[pid=28749] ppid=28747 vsize=6183588 CPUtime=203.71 cores=0-7
/proc/28749/stat : 28749 (cryptominisat-2) R 28747 28749 28749 0 -1 4202496 11301324 0 0 0 18613 1758 0 0 25 0 1 0 625014192 6331994112 1469159 33554432000 4194304 4677096 140733904257840 18446744073709551615 4549546 0 0 16781312 2 0 0 0 17 0 0 0 10
/proc/28749/statm: 1545897 1469159 390 118 0 1540856 0
[pid=28755] ppid=28707 vsize=679948 CPUtime=0.21 cores=0-7
/proc/28755/stat : 28755 (python) S 28707 28707 27951 0 -1 4202560 989 0 0 0 9 12 0 0 18 0 1 0 625024465 696266752 85428 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 12663682347514691844 0 0 17 0 0 0 0
/proc/28755/statm: 169987 85428 339 364 0 105881 0
[pid=28757] ppid=28755 vsize=335840 CPUtime=103.27 cores=0-7
/proc/28757/stat : 28757 (glucose_static) R 28755 28757 28757 0 -1 4202496 79797 0 0 0 10295 32 0 0 25 0 1 0 625024466 343900160 79475 33554432000 134512640 135217611 4287090128 18446744073709551615 134527845 0 0 16781312 8388611 0 0 0 17 6 0 0 0
/proc/28757/statm: 83960 79475 78 173 0 83785 0
[pid=28707/tid=28727] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28727/stat : 28727 (python) S 28705 28707 27951 0 -1 4202560 26 933 0 0 0 0 0 0 25 0 10 0 625014165 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 4 0 0 0
[pid=28707/tid=28730] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28730/stat : 28730 (python) S 28705 28707 27951 0 -1 4202560 27 933 0 0 0 0 0 0 18 0 10 0 625014168 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 4 0 0 0
[pid=28707/tid=28733] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28733/stat : 28733 (python) S 28705 28707 27951 0 -1 4202560 22 933 0 0 0 0 0 0 20 0 10 0 625014172 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 4 0 0 0
[pid=28707/tid=28736] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28736/stat : 28736 (python) S 28705 28707 27951 0 -1 4202560 20 933 0 0 0 0 0 0 20 0 10 0 625014176 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 4 0 0 0
[pid=28707/tid=28739] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28739/stat : 28739 (python) S 28705 28707 27951 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 10 0 625014180 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 4 0 0 0
[pid=28707/tid=28742] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28742/stat : 28742 (python) S 28705 28707 27951 0 -1 4202560 24 933 0 0 0 0 0 0 17 0 10 0 625014184 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 0 0 0 0
[pid=28707/tid=28745] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28745/stat : 28745 (python) S 28705 28707 27951 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 10 0 625014188 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 6 0 0 0
[pid=28707/tid=28748] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28748/stat : 28748 (python) S 28705 28707 27951 0 -1 4202560 28 933 0 0 0 0 0 0 22 0 10 0 625014192 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 0 0 0 0
[pid=28707/tid=28756] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28756/stat : 28756 (python) S 28705 28707 27951 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 10 0 625024466 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2132.65
Current children cumulated vsize (KiB) 31334024

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+689.302 s]
/proc/loadavg: 7.91 4.70 3.70 10/293 28759
/proc/meminfo: memFree=409308/32951124 swapFree=66735608/67111528
[pid=28707] ppid=28705 vsize=755728 CPUtime=482.07 cores=0-7
/proc/28707/stat : 28707 (python) S 28705 28707 27951 0 -1 4202496 298200 933 1 0 48148 59 0 0 19 0 10 0 624965966 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469892555 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/28707/statm: 188932 86409 1313 364 0 124826 0
[pid=28726] ppid=28707 vsize=597900 CPUtime=0.77 cores=0-7
/proc/28726/stat : 28726 (python) S 28707 28707 27951 0 -1 4202560 1285 0 0 0 49 28 0 0 15 0 1 0 625014164 612249600 85403 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085903 0 0 16781312 514 15947681691418460420 0 0 17 1 0 0 0
/proc/28726/statm: 149475 85403 344 364 0 85393 0
[pid=28728] ppid=28726 vsize=6373428 CPUtime=207.31 cores=0-7
/proc/28728/stat : 28728 (cryptominisat-2) R 28726 28728 28728 0 -1 4202496 12843745 0 0 0 18734 1997 0 0 25 0 1 0 625014165 6526390272 1527025 33554432000 4194304 4677096 140735960247520 18446744073709551615 252457763143 0 0 16781312 2 0 0 0 17 7 0 0 7
/proc/28728/statm: 1593357 1527025 390 118 0 1588316 0
[pid=28729] ppid=28707 vsize=608156 CPUtime=0.42 cores=0-7
/proc/28729/stat : 28729 (python) S 28707 28707 27951 0 -1 4202560 1159 0 0 0 22 20 0 0 18 0 2 0 625014168 622751744 85428 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/28729/statm: 152039 85428 364 364 0 87954 0
[pid=28731] ppid=28729 vsize=147292 CPUtime=206.52 cores=0-7
/proc/28731/stat : 28731 (lingeling) R 28729 28731 28731 0 -1 4202496 95490 0 0 0 20636 16 0 0 25 0 1 0 625014169 150827008 35993 33554432000 4194304 4352236 140735691717632 18446744073709551615 4276793 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/28731/statm: 36823 35993 129 39 0 35903 0
[pid=28729/tid=28753] ppid=28707 vsize=608156 CPUtime=0 cores=0-7
/proc/28729/task/28753/stat : 28753 (python) S 28707 28707 27951 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 625024189 622751744 85428 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 514 18446604453848562752 0 0 -1 5 0 0 0
[pid=28732] ppid=28707 vsize=618412 CPUtime=0.62 cores=0-7
/proc/28732/stat : 28732 (python) S 28707 28707 27951 0 -1 4202560 1239 0 0 0 38 24 0 0 15 0 1 0 625014171 633253888 85407 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 1163186549458698500 0 0 17 2 0 0 0
/proc/28732/statm: 154603 85407 340 364 0 90515 0
[pid=28734] ppid=28732 vsize=4363860 CPUtime=206.74 cores=0-7
/proc/28734/stat : 28734 (cryptominisat-2) R 28732 28734 28734 0 -1 4202496 7721095 0 0 0 19156 1518 0 0 25 0 1 0 625014172 4468592640 1023347 33554432000 4194304 4677096 140736821785488 18446744073709551615 4366630 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/28734/statm: 1090965 1023348 390 118 0 1085924 0
[pid=28735] ppid=28707 vsize=628668 CPUtime=0.41 cores=0-7
/proc/28735/stat : 28735 (python) S 28707 28707 27951 0 -1 4202560 1001 0 0 0 21 20 0 0 18 0 1 0 625014175 643756032 85410 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 12663682347514691844 0 0 17 6 0 0 0
/proc/28735/statm: 157167 85410 339 364 0 93076 0
[pid=28737] ppid=28735 vsize=162284 CPUtime=203.82 cores=0-7
/proc/28737/stat : 28737 (lingeling) R 28735 28737 28737 0 -1 4202496 90046 0 0 0 20366 16 0 0 25 0 1 0 625014176 166178816 39742 33554432000 4194304 4352236 140734584733456 18446744073709551615 4319064 0 0 16781312 17506 0 0 0 17 3 0 0 0
/proc/28737/statm: 40571 39742 129 39 0 39651 0
[pid=28738] ppid=28707 vsize=638924 CPUtime=0.21 cores=0-7
/proc/28738/stat : 28738 (python) S 28707 28707 27951 0 -1 4202560 1143 0 0 0 10 11 0 0 17 0 2 0 625014179 654258176 85440 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/28738/statm: 159731 85440 364 364 0 95637 0
[pid=28740] ppid=28738 vsize=994136 CPUtime=102.51 cores=0-7
/proc/28740/stat : 28740 (cryptominisat-2) T 28738 28740 28740 0 -1 4202496 224422 0 0 0 10215 36 0 0 25 0 1 0 625014180 1017995264 182158 33554432000 4194304 4677096 140736653875648 18446744073709551615 4336125 0 0 16781312 2 18446744071562691360 0 0 17 1 0 0 0
/proc/28740/statm: 248534 182158 387 118 0 243493 0
[pid=28738/tid=28754] ppid=28707 vsize=638924 CPUtime=0 cores=0-7
/proc/28738/task/28754/stat : 28754 (python) S 28707 28707 27951 0 -1 4202560 50 0 0 0 0 0 0 0 17 0 2 0 625024461 654258176 85440 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 514 18446604453870898048 0 0 -1 1 0 0 0
[pid=28741] ppid=28707 vsize=649180 CPUtime=0.66 cores=0-7
/proc/28741/stat : 28741 (python) S 28707 28707 27951 0 -1 4202560 1328 0 0 0 39 27 0 0 15 0 1 0 625014183 664760320 85419 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 7925147875909009668 0 0 17 4 0 0 0
/proc/28741/statm: 162295 85419 340 364 0 98198 0
[pid=28743] ppid=28741 vsize=5808248 CPUtime=206.47 cores=0-7
/proc/28743/stat : 28743 (cryptominisat-2) R 28741 28743 28743 0 -1 4202496 12641382 0 0 0 18403 2244 0 0 25 0 1 0 625014184 5947645952 1387008 33554432000 4194304 4677096 140736741271920 18446744073709551615 4364705 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/28743/statm: 1452062 1387010 390 118 0 1447021 0
[pid=28744] ppid=28707 vsize=659436 CPUtime=0.42 cores=0-7
/proc/28744/stat : 28744 (python) S 28707 28707 27951 0 -1 4202560 1006 0 0 0 23 19 0 0 18 0 1 0 625014187 675262464 85422 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 12663682347514691842 0 0 17 6 0 0 0
/proc/28744/statm: 164859 85422 339 364 0 100759 0
[pid=28746] ppid=28744 vsize=459304 CPUtime=205.64 cores=0-7
/proc/28746/stat : 28746 (glucose_static) R 28744 28746 28746 0 -1 4202496 110629 0 0 0 20535 29 0 0 25 0 1 0 625014188 470327296 110307 33554432000 134512640 135217611 4289856192 18446744073709551615 134528127 0 0 16781312 8388611 0 0 0 17 4 0 0 0
/proc/28746/statm: 114826 110307 78 173 0 114651 0
[pid=28747] ppid=28707 vsize=669692 CPUtime=0.87 cores=0-7
/proc/28747/stat : 28747 (python) S 28707 28707 27951 0 -1 4202560 1336 0 0 0 56 31 0 0 15 0 1 0 625014191 685764608 85432 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 8441254235940552962 0 0 17 7 0 0 0
/proc/28747/statm: 167423 85432 345 364 0 103320 0
[pid=28749] ppid=28747 vsize=6183588 CPUtime=203.71 cores=0-7
/proc/28749/stat : 28749 (cryptominisat-2) R 28747 28749 28749 0 -1 4202496 11301324 0 0 0 18613 1758 0 0 25 0 1 0 625014192 6331994112 1469159 33554432000 4194304 4677096 140733904257840 18446744073709551615 4549546 0 0 16781312 2 0 0 0 17 0 0 0 10
/proc/28749/statm: 1545897 1469159 390 118 0 1540856 0
[pid=28755] ppid=28707 vsize=679948 CPUtime=0.21 cores=0-7
/proc/28755/stat : 28755 (python) S 28707 28707 27951 0 -1 4202560 989 0 0 0 9 12 0 0 18 0 1 0 625024465 696266752 85428 33554432000 4194304 5683644 140737488177072 18446744073709551615 252458085958 0 0 16781312 514 12663682347514691844 0 0 17 0 0 0 0
/proc/28755/statm: 169987 85428 339 364 0 105881 0
[pid=28757] ppid=28755 vsize=335840 CPUtime=103.27 cores=0-7
/proc/28757/stat : 28757 (glucose_static) R 28755 28757 28757 0 -1 4202496 79797 0 0 0 10295 32 0 0 25 0 1 0 625024466 343900160 79475 33554432000 134512640 135217611 4287090128 18446744073709551615 134527845 0 0 16781312 8388611 0 0 0 17 6 0 0 0
/proc/28757/statm: 83960 79475 78 173 0 83785 0
[pid=28707/tid=28727] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28727/stat : 28727 (python) S 28705 28707 27951 0 -1 4202560 26 933 0 0 0 0 0 0 25 0 10 0 625014165 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 4 0 0 0
[pid=28707/tid=28730] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28730/stat : 28730 (python) S 28705 28707 27951 0 -1 4202560 27 933 0 0 0 0 0 0 18 0 10 0 625014168 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 4 0 0 0
[pid=28707/tid=28733] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28733/stat : 28733 (python) S 28705 28707 27951 0 -1 4202560 22 933 0 0 0 0 0 0 20 0 10 0 625014172 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 4 0 0 0
[pid=28707/tid=28736] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28736/stat : 28736 (python) S 28705 28707 27951 0 -1 4202560 20 933 0 0 0 0 0 0 20 0 10 0 625014176 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 4 0 0 0
[pid=28707/tid=28739] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28739/stat : 28739 (python) S 28705 28707 27951 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 10 0 625014180 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 4 0 0 0
[pid=28707/tid=28742] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28742/stat : 28742 (python) S 28705 28707 27951 0 -1 4202560 24 933 0 0 0 0 0 0 17 0 10 0 625014184 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 0 0 0 0
[pid=28707/tid=28745] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28745/stat : 28745 (python) S 28705 28707 27951 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 10 0 625014188 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 6 0 0 0
[pid=28707/tid=28748] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28748/stat : 28748 (python) S 28705 28707 27951 0 -1 4202560 28 933 0 0 0 0 0 0 22 0 10 0 625014192 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 0 0 0 0
[pid=28707/tid=28756] ppid=28705 vsize=755728 CPUtime=0 cores=0-7
/proc/28707/task/28756/stat : 28756 (python) S 28705 28707 27951 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 10 0 625024466 773865472 86409 33554432000 4194304 5683644 140737488177072 18446744073709551615 252469889457 0 0 16781312 2 18446604453829407104 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2132.65
Current children cumulated vsize (KiB) 31334024

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 28707 and gives
#  childrusage.ru_utime.tv_sec=481
#  childrusage.ru_utime.tv_usec=488802
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=659899
# CPU time returned by wait4() is 482.149
# while last known CPU time is 2132.65
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 689.63
CPU time (s): 2132.65
CPU user time (s): 2053.68
CPU system time (s): 78.97
CPU usage (%): 309.246
Max. virtual memory (cumulated for all children) (KiB): 31748080

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 481.489
system time used= 0.659899
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 299140
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3429
involuntary context switches= 5719

runsolver used 1.90871 second user time and 4.65629 second system time

The end

Launcher Data 

Begin job on node111 at 2011-05-02 16:42:35
IDJOB=3321581
IDBENCH=82386
IDSOLVER=1710
FILE ID=node111/3321581-1304347353
RUNJOBID= node111-1304335887-27965
PBS_JOBID= 13178022
Free space on /tmp= 72956 MiB

SOLVER NAME= borg-sat 11.03.02-industrial
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-two-cities-sequential-15nodes-1000size-3degree-100mindistance-3trucks-10packages-2008seed.040-SAT.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321581-1304347353/watcher-3321581-1304347353 -o /tmp/evaluation-result-3321581-1304347353/solver-3321581-1304347353 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3321581-1304347353.cnf -seed 1796958160 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 74e0fa52e4789615985c73d0a8b105b4
RANDOM SEED=1796958160

node111.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5428.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21405856 kB
Buffers:       1071512 kB
Cached:        3216432 kB
SwapCached:          0 kB
Active:        7353612 kB
Inactive:      3706360 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21405856 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:           91608 kB
Writeback:           0 kB
AnonPages:     6771944 kB
Mapped:          25420 kB
Slab:           395212 kB
PageTables:      24764 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 38328336 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72924 MiB
End job on node111 at 2011-05-02 16:54:07