Trace number 3380784

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-industrial (fixed)? (MO) 17497.3 2216.12

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.025-NOTKNOWN.cnf
MD5SUM3c5c73741ba21d5f39c7f436ba834743
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables95456
Number of clauses483561
Sum of the clauses size1149911
Maximum clause length63
Minimum clause length1
Number of clauses of size 1211
Number of clauses of size 2477300
Number of clauses of size 31500
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 54550

Solver Data 

0.35/0.68	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.35/0.68	c INFO: solving HOME/instance-3380784-1305492135.cnf
0.35/0.68	c INFO: computing task features
1.72/2.06	c INFO: computed variable-clause graph statistics
1.82/2.12	c INFO: computed clause balance statistics
8.42/8.81	c INFO: computed variable balance statistics
13.11/13.47	c INFO: computed small-clause counts
13.21/13.53	c INFO: computed Horn-clause counts
24.01/24.38	c INFO: computed variable graph statistics
24.01/24.38	c INFO: collected features for HOME/instance-3380784-1305492135.cnf in 23.67 s
24.12/24.41	c INFO: running SApperloT@0 for 97 with 4976 remaining (b = 0.13)
24.12/24.44	c INFO: running lingeling-276@0 for 97 with 4976 remaining (b = 0.15)
24.12/24.46	c INFO: running cryptominisat-2.9.0@0 for 582 with 4976 remaining (b = 0.25)
24.12/24.49	c INFO: running lingeling-276@0 for 485 with 4976 remaining (b = 0.22)
24.21/24.52	c INFO: running lingeling-276@0 for 1261 with 4976 remaining (b = 0.32)
24.21/24.55	c INFO: running cryptominisat-2.9.0@0 for 1649 with 4976 remaining (b = 0.39)
24.21/24.57	c INFO: running cryptominisat-2.9.0@0 for 1649 with 4976 remaining (b = 0.33)
24.31/24.60	c INFO: running precosat-570@0 for 388 with 4976 remaining (b = 0.11)
824.00/124.75	c INFO: running lingeling-276@0 for 1261 with 4875 remaining (b = 0.24)
824.00/124.78	c INFO: running cryptominisat-2.9.0@0 for 1940 with 4875 remaining (b = 0.31)
3161.57/417.87	c INFO: running lingeling-276@0 for 1164 with 4582 remaining (b = 0.18)
3944.16/515.99	c INFO: running SApperloT@0 for 97 with 4484 remaining (b = 0.02)
4716.30/612.75	c INFO: running precosat-570@100 for 291 with 4387 remaining (b = 0.10)
4736.92/615.36	c INFO: running precosat-570@0 for 291 with 4385 remaining (b = 0.09)
7028.52/902.66	c INFO: running precosat-570@199 for 194 with 4098 remaining (b = 0.05)
7120.98/914.29	c INFO: running precosat-570@288 for 194 with 4086 remaining (b = 0.05)
8604.59/1100.27	c INFO: running lingeling-276@0 for 97 with 3900 remaining (b = 0.03)
8649.22/1105.89	c INFO: running precosat-570@495 for 1746 with 3894 remaining (b = 0.24)
9398.19/1199.71	c INFO: running precosat-570@485 for 291 with 3800 remaining (b = 0.06)
10121.64/1290.48	c INFO: running cryptominisat-2.9.0@0 for 1843 with 3710 remaining (b = 0.25)
10927.33/1391.58	c INFO: running cryptominisat-2.9.0@0 for 97 with 3609 remaining (b = 0.02)
11701.55/1488.62	c INFO: running precosat-570@1747 for 485 with 3512 remaining (b = 0.07)
11758.91/1495.88	c INFO: running cryptominisat-2.9.0@288 for 291 with 3504 remaining (b = 0.05)
12465.46/1584.42	c INFO: running precosat-570@0 for 388 with 3416 remaining (b = 0.07)
13329.34/1692.86	c INFO: running lingeling-276@489 for 776 with 3307 remaining (b = 0.11)
13543.57/1719.72	c INFO: running lingeling-276@0 for 1164 with 3280 remaining (b = 0.13)
14113.15/1791.28	c INFO: running cryptominisat-2.9.0@1451 for 97 with 3209 remaining (b = 0.03)
14913.76/1891.70	c INFO: running cryptominisat-2.9.0@1548 for 194 with 3108 remaining (b = 0.04)
15614.01/1979.64	c INFO: running precosat-570@1655 for 582 with 3021 remaining (b = 0.06)
15626.07/1981.12	c INFO: running precosat-570@0 for 388 with 3019 remaining (b = 0.07)
16394.72/2077.60	c INFO: running lingeling-276@190 for 291 with 2923 remaining (b = 0.04)
16531.71/2094.87	c INFO: running precosat-570@0 for 97 with 2905 remaining (b = 0.03)
17331.57/2195.25	c INFO: running cryptominisat-2.9.0@1939 for 97 with 2805 remaining (b = 0.03)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3380784-1305492135/watcher-3380784-1305492135 -o /tmp/evaluation-result-3380784-1305492135/solver-3380784-1305492135 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3380784-1305492135.cnf -seed 1302967919 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.53 7.97 7.97 1/407 4363
/proc/meminfo: memFree=22983196/32950928 swapFree=45937200/67111528
[pid=4363] ppid=4361 vsize=15380 CPUtime=0 cores=0-7
/proc/4363/stat : 4363 (python) R 4361 4363 4156 0 -1 4202496 807 0 0 0 0 0 0 0 17 0 1 0 660810275 15749120 693 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500691600 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/4363/statm: 3845 693 332 364 0 332 0

[startup+0.032028 s]
/proc/loadavg: 7.53 7.97 7.97 1/407 4363
/proc/meminfo: memFree=22983196/32950928 swapFree=45937200/67111528
[pid=4363] ppid=4361 vsize=16392 CPUtime=0.01 cores=0-7
/proc/4363/stat : 4363 (python) R 4361 4363 4156 0 -1 4202496 1164 0 0 0 1 0 0 0 17 0 1 0 660810275 16785408 1029 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500689920 0 2147483391 16781312 2 0 0 0 17 2 0 0 0
/proc/4363/statm: 4098 1029 414 364 0 585 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16392

[startup+0.100988 s]
/proc/loadavg: 7.53 7.97 7.97 1/407 4363
/proc/meminfo: memFree=22983196/32950928 swapFree=45937200/67111528
[pid=4363] ppid=4361 vsize=85616 CPUtime=0.06 cores=0-7
/proc/4363/stat : 4363 (python) S 4361 4363 4156 0 -1 4202496 2629 0 0 0 5 1 0 0 17 0 1 0 660810275 87670784 2347 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500689920 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/4363/statm: 21404 2347 713 364 0 1621 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85616

[startup+0.300964 s]
/proc/loadavg: 7.53 7.97 7.97 1/407 4363
/proc/meminfo: memFree=22983196/32950928 swapFree=45937200/67111528
[pid=4363] ppid=4361 vsize=135748 CPUtime=0.19 cores=0-7
/proc/4363/stat : 4363 (python) R 4361 4363 4156 0 -1 4202496 5963 932 1 0 14 5 0 0 18 0 1 0 660810275 139005952 4150 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500687877 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/4363/statm: 33937 4150 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135748

[startup+0.700863 s]
/proc/loadavg: 7.53 7.97 7.97 1/407 4363
/proc/meminfo: memFree=22983196/32950928 swapFree=45937200/67111528
[pid=4363] ppid=4361 vsize=286936 CPUtime=0.41 cores=0-7
/proc/4363/stat : 4363 (python) R 4361 4363 4156 0 -1 4202496 11397 932 1 0 31 10 0 0 17 0 1 0 660810275 293822464 9079 33554432000 4194304 5683644 140736271978032 18446744073709551615 4619584 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/4363/statm: 71734 9080 1761 364 0 7658 0
Current children cumulated CPU time (s) 0.41
Current children cumulated vsize (KiB) 286936

[startup+1.5007 s]
/proc/loadavg: 7.53 7.97 7.97 2/408 4368
/proc/meminfo: memFree=22932224/32950928 swapFree=45937200/67111528
[pid=4363] ppid=4361 vsize=343008 CPUtime=1.22 cores=0-7
/proc/4363/stat : 4363 (python) R 4361 4363 4156 0 -1 4202496 25221 932 1 0 110 12 0 0 18 0 1 0 660810275 351240192 22903 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500352709 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/4363/statm: 85752 22904 1761 364 0 21676 0
Current children cumulated CPU time (s) 1.22
Current children cumulated vsize (KiB) 343008

[startup+3.10038 s]
/proc/loadavg: 7.53 7.97 7.97 2/408 4368
/proc/meminfo: memFree=22861916/32950928 swapFree=45937200/67111528
[pid=4363] ppid=4361 vsize=376772 CPUtime=2.81 cores=0-7
/proc/4363/stat : 4363 (python) R 4361 4363 4156 0 -1 4202496 44860 932 1 0 268 13 0 0 21 0 1 0 660810275 385814528 31578 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500352593 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/4363/statm: 94193 31578 1826 364 0 30117 0
Current children cumulated CPU time (s) 2.81
Current children cumulated vsize (KiB) 376772

[startup+6.30075 s]
/proc/loadavg: 7.08 7.87 7.94 2/408 4368
/proc/meminfo: memFree=22861516/32950928 swapFree=45937200/67111528
[pid=4363] ppid=4361 vsize=376772 CPUtime=6.01 cores=0-7
/proc/4363/stat : 4363 (python) R 4361 4363 4156 0 -1 4202496 44860 932 1 0 588 13 0 0 25 0 1 0 660810275 385814528 31578 33554432000 4194304 5683644 140736271978032 18446744073709551615 47237969874708 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/4363/statm: 94193 31578 1826 364 0 30117 0
Current children cumulated CPU time (s) 6.01
Current children cumulated vsize (KiB) 376772

[startup+12.7005 s]
/proc/loadavg: 6.60 7.75 7.90 2/408 4368
/proc/meminfo: memFree=22861780/32950928 swapFree=45937200/67111528
[pid=4363] ppid=4361 vsize=376772 CPUtime=12.41 cores=0-7
/proc/4363/stat : 4363 (python) R 4361 4363 4156 0 -1 4202496 44861 932 1 0 1228 13 0 0 25 0 1 0 660810275 385814528 31579 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500350620 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/4363/statm: 94193 31579 1827 364 0 30117 0
Current children cumulated CPU time (s) 12.41
Current children cumulated vsize (KiB) 376772

[startup+25.5018 s]
/proc/loadavg: 5.36 7.42 7.79 9/432 4392
/proc/meminfo: memFree=22589180/32950928 swapFree=45937200/67111528
[pid=4363] ppid=4361 vsize=516860 CPUtime=24.32 cores=0-7
/proc/4363/stat : 4363 (python) S 4361 4363 4156 0 -1 4202496 50638 932 1 0 2415 17 0 0 20 0 9 0 660810275 529264640 29798 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512518603 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4363/statm: 129215 29798 1889 364 0 65112 0
[pid=4369] ppid=4363 vsize=369288 CPUtime=0 cores=0-7
/proc/4369/stat : 4369 (python) S 4363 4363 4156 0 -1 4202560 809 0 0 0 0 0 0 0 18 0 1 0 660812716 378150912 28233 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500711951 0 0 16781312 514 11408718196685930495 0 0 17 1 0 0 0
/proc/4369/statm: 92322 28233 358 364 0 28240 0
[pid=4371] ppid=4369 vsize=16512 CPUtime=1.07 cores=0-7
/proc/4371/stat : 4371 (SApperloT-base) R 4369 4371 4371 0 -1 4202496 3680 0 0 0 106 1 0 0 25 0 1 0 660812716 16908288 3382 33554432000 134512640 135674940 4290181168 18446744073709551615 134551170 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/4371/statm: 4128 3382 99 284 0 3842 0
[pid=4372] ppid=4363 vsize=379544 CPUtime=0 cores=0-7
/proc/4372/stat : 4372 (python) S 4363 4363 4156 0 -1 4202560 748 0 0 0 0 0 0 0 18 0 1 0 660812719 388653056 28226 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 16825057060889559039 0 0 17 4 0 0 0
/proc/4372/statm: 94886 28226 347 364 0 30801 0
[pid=4374] ppid=4372 vsize=17944 CPUtime=1.04 cores=0-7
/proc/4374/stat : 4374 (lingeling) R 4372 4374 4374 0 -1 4202496 7700 0 0 0 103 1 0 0 25 0 1 0 660812719 18374656 3641 33554432000 4194304 4352236 140736121188208 18446744073709551615 4319146 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/4374/statm: 4486 3641 121 39 0 3566 0
[pid=4375] ppid=4363 vsize=389800 CPUtime=0 cores=0-7
/proc/4375/stat : 4375 (python) S 4363 4363 4156 0 -1 4202560 751 0 0 0 0 0 0 0 15 0 1 0 660812721 399155200 28230 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 7488105198007582978 0 0 17 6 0 0 0
/proc/4375/statm: 97450 28230 347 364 0 33362 0
[pid=4377] ppid=4375 vsize=111668 CPUtime=1.01 cores=0-7
/proc/4377/stat : 4377 (cryptominisat-2) R 4375 4377 4377 0 -1 4202496 20997 0 0 0 99 2 0 0 24 0 1 0 660812722 114348032 18539 33554432000 4194304 4677096 140734782528880 18446744073709551615 4236529 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/4377/statm: 27917 18543 361 118 0 22876 0
[pid=4378] ppid=4363 vsize=400056 CPUtime=0 cores=0-7
/proc/4378/stat : 4378 (python) S 4363 4363 4156 0 -1 4202560 741 0 0 0 0 0 0 0 19 0 1 0 660812724 409657344 28230 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 2144790948318543871 0 0 17 4 0 0 0
/proc/4378/statm: 100014 28230 343 364 0 35923 0
[pid=4380] ppid=4378 vsize=18216 CPUtime=0.98 cores=0-7
/proc/4380/stat : 4380 (lingeling) R 4378 4380 4380 0 -1 4202496 7770 0 0 0 97 1 0 0 23 0 1 0 660812725 18653184 3710 33554432000 4194304 4352236 140734301761616 18446744073709551615 4319072 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/4380/statm: 4554 3710 121 39 0 3634 0
[pid=4381] ppid=4363 vsize=410312 CPUtime=0 cores=0-7
/proc/4381/stat : 4381 (python) S 4363 4363 4156 0 -1 4202560 753 0 0 0 0 0 0 0 18 0 1 0 660812727 420159488 28234 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 2144787100027846655 0 0 17 1 0 0 0
/proc/4381/statm: 102578 28234 343 364 0 38484 0
[pid=4383] ppid=4381 vsize=18244 CPUtime=0.96 cores=0-7
/proc/4383/stat : 4383 (lingeling) R 4381 4383 4383 0 -1 4202496 7782 0 0 0 95 1 0 0 22 0 1 0 660812727 18681856 3717 33554432000 4194304 4352236 140735529452880 18446744073709551615 4319146 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/4383/statm: 4561 3717 121 39 0 3641 0
[pid=4384] ppid=4363 vsize=420568 CPUtime=0 cores=0-7
/proc/4384/stat : 4384 (python) S 4363 4363 4156 0 -1 4202560 778 0 0 0 0 0 0 0 16 0 1 0 660812730 430661632 28242 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 18186366530407596290 0 0 17 1 0 0 0
/proc/4384/statm: 105142 28242 347 364 0 41045 0
[pid=4386] ppid=4384 vsize=105420 CPUtime=0.92 cores=0-7
/proc/4386/stat : 4386 (cryptominisat-2) R 4384 4386 4386 0 -1 4202496 19652 0 0 0 89 3 0 0 25 0 1 0 660812730 107950080 17191 33554432000 4194304 4677096 140733558936864 18446744073709551615 254500717674 0 0 16781312 2 0 0 0 17 7 0 0 0
/proc/4386/statm: 26355 17191 361 118 0 21314 0
[pid=4387] ppid=4363 vsize=496360 CPUtime=0 cores=0-7
/proc/4387/stat : 4387 (python) S 4363 4363 4156 0 -1 4202560 784 0 0 0 0 0 0 0 15 0 1 0 660812733 508272640 28248 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 14099213529127026947 0 0 17 1 0 0 0
/proc/4387/statm: 124090 28248 347 364 0 59990 0
[pid=4389] ppid=4387 vsize=101356 CPUtime=0.88 cores=0-7
/proc/4389/stat : 4389 (cryptominisat-2) R 4387 4389 4389 0 -1 4202496 18870 0 0 0 86 2 0 0 23 0 1 0 660812735 103788544 16410 33554432000 4194304 4677096 140736963932960 18446744073709551615 4236529 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/4389/statm: 25339 16411 361 118 0 20298 0
[pid=4390] ppid=4363 vsize=506616 CPUtime=0 cores=0-7
/proc/4390/stat : 4390 (python) S 4363 4363 4156 0 -1 4202560 808 0 0 0 0 0 0 0 20 0 1 0 660812735 518774784 28254 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 12089654543860629503 0 0 17 2 0 0 0
/proc/4390/statm: 126654 28254 349 364 0 62551 0
[pid=4392] ppid=4390 vsize=63588 CPUtime=0.88 cores=0-7
/proc/4392/stat : 4392 (precosat) R 4390 4392 4392 0 -1 4202496 17103 0 0 0 86 2 0 0 25 0 1 0 660812736 65114112 15331 33554432000 4194304 4890832 140737357533232 18446744073709551615 4294179 0 0 16781312 17442 0 0 0 17 1 0 0 0
/proc/4392/statm: 15897 15331 81 171 0 15724 0
[pid=4363/tid=4370] ppid=4361 vsize=516860 CPUtime=0 cores=0-7
/proc/4363/task/4370/stat : 4370 (python) S 4361 4363 4156 0 -1 4202560 25 932 0 0 0 0 0 0 25 0 9 0 660812716 529264640 29798 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 0 0 0 0
[pid=4363/tid=4373] ppid=4361 vsize=516860 CPUtime=0 cores=0-7
/proc/4363/task/4373/stat : 4373 (python) S 4361 4363 4156 0 -1 4202560 21 932 0 0 0 0 0 0 25 0 9 0 660812719 529264640 29798 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 0 0 0 0
[pid=4363/tid=4376] ppid=4361 vsize=516860 CPUtime=0 cores=0-7
/proc/4363/task/4376/stat : 4376 (python) S 4361 4363 4156 0 -1 4202560 24 932 0 0 0 0 0 0 20 0 9 0 660812722 529264640 29798 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 0 0 0 0
[pid=4363/tid=4379] ppid=4361 vsize=516860 CPUtime=0 cores=0-7
/proc/4363/task/4379/stat : 4379 (python) S 4361 4363 4156 0 -1 4202560 22 932 0 0 0 0 0 0 20 0 9 0 660812724 529264640 29798 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 6 0 0 0
[pid=4363/tid=4382] ppid=4361 vsize=516860 CPUtime=0 cores=0-7
/proc/4363/task/4382/stat : 4382 (python) S 4361 4363 4156 0 -1 4202560 23 932 0 0 0 0 0 0 19 0 9 0 660812727 529264640 29798 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 4 0 0 0
[pid=4363/tid=4385] ppid=4361 vsize=516860 CPUtime=0 cores=0-7
/proc/4363/task/4385/stat : 4385 (python) S 4361 4363 4156 0 -1 4202560 20 932 0 0 0 0 0 0 19 0 9 0 660812730 529264640 29798 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 7 0 0 0
[pid=4363/tid=4388] ppid=4361 vsize=516860 CPUtime=0 cores=0-7
/proc/4363/task/4388/stat : 4388 (python) S 4361 4363 4156 0 -1 4202560 35 932 0 0 0 0 0 0 19 0 9 0 660812733 529264640 29798 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 1 0 0 0
[pid=4363/tid=4391] ppid=4361 vsize=516860 CPUtime=0 cores=0-7
/proc/4363/task/4391/stat : 4391 (python) S 4361 4363 4156 0 -1 4202560 24 932 0 0 0 0 0 0 21 0 9 0 660812736 529264640 29798 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 32.06
Current children cumulated vsize (KiB) 4342352

[startup+51.1017 s]
/proc/loadavg: 6.26 7.47 7.80 9/432 4393
/proc/meminfo: memFree=21030316/32950928 swapFree=45937200/67111528
[pid=4363] ppid=4361 vsize=516860 CPUtime=24.32 cores=0-7
/proc/4363/stat : 4363 (python) S 4361 4363 4156 0 -1 4202496 50638 932 1 0 2415 17 0 0 20 0 9 0 660810275 529264640 29798 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512518603 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/4363/statm: 129215 29798 1889 364 0 65112 0
[pid=4369] ppid=4363 vsize=369288 CPUtime=0.07 cores=0-7
/proc/4369/stat : 4369 (python) S 4363 4363 4156 0 -1 4202560 999 0 0 0 5 2 0 0 18 0 1 0 660812716 378150912 28256 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500711951 0 0 16781312 514 8024957139137036545 0 0 17 1 0 0 0
/proc/4369/statm: 92322 28256 381 364 0 28240 0
[pid=4371] ppid=4369 vsize=34176 CPUtime=26.67 cores=0-7
/proc/4371/stat : 4371 (SApperloT-base) R 4369 4371 4371 0 -1 4202496 8176 0 0 0 2665 2 0 0 25 0 1 0 660812716 34996224 7813 33554432000 134512640 135674940 4290181168 18446744073709551615 134551972 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/4371/statm: 8544 7813 99 284 0 8258 0
[pid=4372] ppid=4363 vsize=379544 CPUtime=0.07 cores=0-7
/proc/4372/stat : 4372 (python) S 4363 4363 4156 0 -1 4202560 923 0 0 0 3 4 0 0 18 0 1 0 660812719 388653056 28249 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 8024957139137036546 0 0 17 4 0 0 0
/proc/4372/statm: 94886 28249 370 364 0 30801 0
[pid=4374] ppid=4372 vsize=38984 CPUtime=26.64 cores=0-7
/proc/4374/stat : 4374 (lingeling) R 4372 4374 4374 0 -1 4202496 13095 0 0 0 2663 1 0 0 25 0 1 0 660812719 39919616 8916 33554432000 4194304 4352236 140736121188208 18446744073709551615 4208066 0 0 16781312 17506 0 0 0 17 5 0 0 0
/proc/4374/statm: 9746 8916 129 39 0 8826 0
[pid=4375] ppid=4363 vsize=389800 CPUtime=0.09 cores=0-7
/proc/4375/stat : 4375 (python) S 4363 4363 4156 0 -1 4202560 967 0 0 0 4 5 0 0 15 0 1 0 660812721 399155200 28253 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 8024957139137036546 0 0 17 6 0 0 0
/proc/4375/statm: 97450 28253 370 364 0 33362 0
[pid=4377] ppid=4375 vsize=729028 CPUtime=26.61 cores=0-7
/proc/4377/stat : 4377 (cryptominisat-2) R 4375 4377 4377 0 -1 4202496 142693 0 0 0 2645 16 0 0 25 0 1 0 660812722 746524672 133198 33554432000 4194304 4677096 140734782528880 18446744073709551615 4336232 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/4377/statm: 182257 133198 387 118 0 177216 0
[pid=4378] ppid=4363 vsize=400056 CPUtime=0.07 cores=0-7
/proc/4378/stat : 4378 (python) S 4363 4363 4156 0 -1 4202560 960 0 0 0 4 3 0 0 18 0 1 0 660812724 409657344 28257 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 8024957139137036546 0 0 17 4 0 0 0
/proc/4378/statm: 100014 28257 370 364 0 35923 0

################
# More data... #
################

[pid=4377] ppid=4375 vsize=4179536 CPUtime=1513.84 cores=0-7
/proc/4377/stat : 4377 (cryptominisat-2) R 4375 4377 4377 0 -1 4202496 1569105 0 0 0 151059 325 0 0 25 0 1 0 660812722 4279844864 967213 33554432000 4194304 4677096 140734782528880 18446744073709551615 4318908 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/4377/statm: 1044884 967213 393 118 0 1039843 0
[pid=4375/tid=4422] ppid=4363 vsize=389800 CPUtime=0 cores=0-7
/proc/4375/task/4422/stat : 4422 (python) S 4363 4363 4156 0 -1 4202560 44 0 0 0 0 0 0 0 17 0 2 0 660871548 399155200 28284 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604449117099904 0 0 -1 6 0 0 0
[pid=4378] ppid=4363 vsize=400056 CPUtime=3.17 cores=0-7
/proc/4378/stat : 4378 (python) S 4363 4363 4156 0 -1 4202560 1177 0 0 0 161 156 0 0 18 0 2 0 660812724 409657344 28287 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/4378/statm: 100014 28287 399 364 0 35923 0
[pid=4380] ppid=4378 vsize=125444 CPUtime=983.72 cores=0-7
/proc/4380/stat : 4380 (lingeling) R 4378 4380 4380 0 -1 4202496 348201 0 0 0 98320 52 0 0 25 0 1 0 660812725 128454656 30563 33554432000 4194304 4352236 140734301761616 18446744073709551615 4278664 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/4380/statm: 31361 30563 129 39 0 30441 0
[pid=4378/tid=4416] ppid=4363 vsize=400056 CPUtime=0 cores=0-7
/proc/4378/task/4416/stat : 4416 (python) S 4363 4363 4156 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 660861871 409657344 28287 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604449117099008 0 0 -1 4 0 0 0
[pid=4381] ppid=4363 vsize=410312 CPUtime=4.01 cores=0-7
/proc/4381/stat : 4381 (python) S 4363 4363 4156 0 -1 4202560 1179 0 0 0 200 201 0 0 19 0 2 0 660812727 420159488 28291 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/4381/statm: 102578 28291 399 364 0 38484 0
[pid=4383] ppid=4381 vsize=124696 CPUtime=1262.46 cores=0-7
/proc/4383/stat : 4383 (lingeling) T 4381 4383 4383 0 -1 4202496 361032 0 0 0 126204 42 0 0 25 0 1 0 660812727 127688704 30376 33554432000 4194304 4352236 140735529452880 18446744073709551615 4278597 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/4383/statm: 31174 30376 129 39 0 30254 0
[pid=4381/tid=4459] ppid=4363 vsize=410312 CPUtime=0 cores=0-7
/proc/4381/task/4459/stat : 4459 (python) S 4363 4363 4156 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 660939320 420159488 28291 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604449117098112 0 0 -1 0 0 0 0
[pid=4384] ppid=4363 vsize=486104 CPUtime=6.73 cores=0-7
/proc/4384/stat : 4384 (python) S 4363 4363 4156 0 -1 4202560 2187 0 0 0 367 306 0 0 17 0 2 0 660812730 497770496 28297 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/4384/statm: 121526 28297 400 364 0 57429 0
[pid=4386] ppid=4384 vsize=4179308 CPUtime=1654.87 cores=0-7
/proc/4386/stat : 4386 (cryptominisat-2) T 4384 4386 4386 0 -1 4202496 1592907 0 0 0 165121 366 0 0 25 0 1 0 660812730 4279611392 985185 33554432000 4194304 4677096 140733558936864 18446744073709551615 4334257 0 0 16781312 2 18446744071562691360 0 0 17 1 0 0 0
/proc/4386/statm: 1044827 985185 393 118 0 1039786 0
[pid=4384/tid=4484] ppid=4363 vsize=486104 CPUtime=0 cores=0-7
/proc/4384/task/4484/stat : 4484 (python) S 4363 4363 4156 0 -1 4202560 45 0 0 0 0 0 0 0 17 0 2 0 660982244 497770496 28297 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604449608581248 0 0 -1 0 0 0 0
[pid=4387] ppid=4363 vsize=496360 CPUtime=6.49 cores=0-7
/proc/4387/stat : 4387 (python) S 4363 4363 4156 0 -1 4202560 2282 0 0 0 348 301 0 0 16 0 2 0 660812733 508272640 28302 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/4387/statm: 124090 28302 400 364 0 59990 0
[pid=4389] ppid=4387 vsize=4234144 CPUtime=1658.04 cores=0-7
/proc/4389/stat : 4389 (cryptominisat-2) T 4387 4389 4389 0 -1 4202496 1559988 0 0 0 165514 290 0 0 25 0 1 0 660812735 4335763456 977891 33554432000 4194304 4677096 140736963932960 18446744073709551615 4320251 0 0 16781312 2 18446744071562691360 0 0 17 4 0 0 0
/proc/4389/statm: 1058536 977891 393 118 0 1053495 0
[pid=4387/tid=4482] ppid=4363 vsize=496360 CPUtime=0 cores=0-7
/proc/4387/task/4482/stat : 4482 (python) S 4363 4363 4156 0 -1 4202560 47 0 0 0 0 0 0 0 16 0 2 0 660979559 508272640 28302 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604447137601600 0 0 -1 4 0 0 0
[pid=4390] ppid=4363 vsize=506616 CPUtime=6.4 cores=0-7
/proc/4390/stat : 4390 (python) S 4363 4363 4156 0 -1 4202560 1236 0 0 0 321 319 0 0 18 0 2 0 660812735 518774784 28311 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4390/statm: 126654 28311 405 364 0 62551 0
[pid=4392] ppid=4390 vsize=279336 CPUtime=1978.72 cores=0-7
/proc/4392/stat : 4392 (precosat) R 4390 4392 4392 0 -1 4202496 163656 0 0 0 197846 26 0 0 25 0 1 0 660812736 286040064 69234 33554432000 4194304 4890832 140737357533232 18446744073709551615 4297357 0 0 16781312 17442 0 0 0 17 4 0 0 0
/proc/4392/statm: 69834 69234 86 171 0 69661 0
[pid=4390/tid=4412] ppid=4363 vsize=506616 CPUtime=0 cores=0-7
/proc/4390/task/4412/stat : 4412 (python) S 4363 4363 4156 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 660852059 518774784 28311 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604447137600704 0 0 -1 1 0 0 0
[pid=4398] ppid=4363 vsize=516872 CPUtime=4.02 cores=0-7
/proc/4398/stat : 4398 (python) S 4363 4363 4156 0 -1 4202560 1194 0 0 0 205 197 0 0 18 0 2 0 660822750 529276928 28309 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/4398/statm: 129218 28309 399 364 0 65112 0
[pid=4400] ppid=4398 vsize=130340 CPUtime=1264.82 cores=0-7
/proc/4400/stat : 4400 (lingeling) T 4398 4400 4400 0 -1 4202496 403957 0 0 0 126437 45 0 0 25 0 1 0 660822751 133468160 31786 33554432000 4194304 4352236 140734322212912 18446744073709551615 4278811 0 0 16781312 17506 18446744071562691360 0 0 17 6 0 0 0
/proc/4400/statm: 32585 31786 129 39 0 31665 0
[pid=4398/tid=4466] ppid=4363 vsize=516872 CPUtime=0 cores=0-7
/proc/4398/task/4466/stat : 4466 (python) S 4363 4363 4156 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 660949431 529276928 28309 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604450214376512 0 0 -1 6 0 0 0
[pid=4401] ppid=4363 vsize=527128 CPUtime=7.56 cores=0-7
/proc/4401/stat : 4401 (python) S 4363 4363 4156 0 -1 4202560 2240 0 0 0 404 352 0 0 16 0 2 0 660822753 539779072 28314 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/4401/statm: 131782 28314 400 364 0 67673 0
[pid=4403] ppid=4401 vsize=4325936 CPUtime=1945.33 cores=0-7
/proc/4403/stat : 4403 (cryptominisat-2) T 4401 4403 4403 0 -1 4202496 1676982 0 0 0 194170 363 0 0 25 0 1 0 660822754 4429758464 1006631 33554432000 4194304 4677096 140735716210320 18446744073709551615 4355536 0 0 16781312 2 18446744071562691360 0 0 17 3 0 0 0
/proc/4403/statm: 1081484 1006631 393 118 0 1076443 0
[pid=4401/tid=4498] ppid=4363 vsize=527128 CPUtime=0 cores=0-7
/proc/4401/task/4498/stat : 4498 (python) S 4363 4363 4156 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 661018033 539779072 28314 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604450214375616 0 0 -1 0 0 0 0
[pid=4417] ppid=4363 vsize=537384 CPUtime=0.32 cores=0-7
/proc/4417/stat : 4417 (python) S 4363 4363 4156 0 -1 4202560 1074 0 0 0 17 15 0 0 19 0 2 0 660861874 550281216 28419 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4417/statm: 134346 28419 399 364 0 70234 0
[pid=4419] ppid=4417 vsize=80088 CPUtime=98.88 cores=0-7
/proc/4419/stat : 4419 (SApperloT-base) T 4417 4419 4419 0 -1 4202496 19724 0 0 0 9885 3 0 0 25 0 1 0 660861875 82010112 19236 33554432000 134512640 135674940 4288881504 18446744073709551615 134552072 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/4419/statm: 20022 19236 99 284 0 19736 0
[pid=4417/tid=4423] ppid=4363 vsize=537384 CPUtime=0 cores=0-7
/proc/4417/task/4423/stat : 4423 (python) S 4363 4363 4156 0 -1 4202560 119 0 0 0 0 0 0 0 19 0 2 0 660871808 550281216 28419 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604450980582656 0 0 -1 0 0 0 0
[pid=4424] ppid=4363 vsize=547640 CPUtime=4.06 cores=0-7
/proc/4424/stat : 4424 (python) S 4363 4363 4156 0 -1 4202560 1053 0 0 0 202 204 0 0 18 0 2 0 660871811 560783360 28460 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4424/statm: 136910 28460 399 364 0 72795 0
[pid=4426] ppid=4424 vsize=231476 CPUtime=1262.07 cores=0-7
/proc/4426/stat : 4426 (precosat) T 4424 4426 4426 0 -1 4202496 132719 0 0 0 126185 22 0 0 25 0 1 0 660871812 237031424 57256 33554432000 4194304 4890832 140737226575216 18446744073709551615 4234992 0 0 16781312 17442 18446744071562691360 0 0 17 1 0 0 0
/proc/4426/statm: 57869 57256 86 171 0 57696 0
[pid=4424/tid=4435] ppid=4363 vsize=547640 CPUtime=0 cores=0-7
/proc/4424/task/4435/stat : 4435 (python) S 4363 4363 4156 0 -1 4202560 80 0 0 0 0 0 0 0 18 0 2 0 660901701 560783360 28460 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604445337564160 0 0 -1 1 0 0 0
[pid=4441] ppid=4363 vsize=557896 CPUtime=0.78 cores=0-7
/proc/4441/stat : 4441 (python) S 4363 4363 4156 0 -1 4202560 1197 0 0 0 41 37 0 0 18 0 2 0 660920302 571285504 28446 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4441/statm: 139474 28446 399 364 0 75356 0
[pid=4443] ppid=4441 vsize=101680 CPUtime=237.45 cores=0-7
/proc/4443/stat : 4443 (lingeling) R 4441 4443 4443 0 -1 4202496 42249 0 0 0 23740 5 0 0 25 0 1 0 660920303 104120320 24590 33554432000 4194304 4352236 140735570701056 18446744073709551615 4280960 0 0 16781312 17506 0 0 0 17 1 0 0 0
/proc/4443/statm: 25420 24590 129 39 0 24500 0
[pid=4441/tid=4456] ppid=4363 vsize=557896 CPUtime=0 cores=0-7
/proc/4441/task/4456/stat : 4456 (python) S 4363 4363 4156 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 660930243 571285504 28446 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604447021089728 0 0 -1 1 0 0 0
[pid=4467] ppid=4363 vsize=568152 CPUtime=3.36 cores=0-7
/proc/4467/stat : 4467 (python) S 4363 4363 4156 0 -1 4202560 2195 0 0 0 191 145 0 0 15 0 2 0 660949433 581787648 28451 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/4467/statm: 142038 28451 400 364 0 77917 0
[pid=4469] ppid=4467 vsize=3956568 CPUtime=705.86 cores=0-7
/proc/4469/stat : 4469 (cryptominisat-2) R 4467 4469 4469 0 -1 4202496 1255351 0 0 0 70350 236 0 0 25 0 1 0 660949434 4051525632 918731 33554432000 4194304 4677096 140733730208912 18446744073709551615 4334547 0 0 16781312 2 0 0 0 17 7 0 0 0
/proc/4469/statm: 989142 918731 393 118 0 984101 0
[pid=4467/tid=4473] ppid=4363 vsize=568152 CPUtime=0 cores=0-7
/proc/4467/task/4473/stat : 4473 (python) S 4363 4363 4156 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 660959861 581787648 28451 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604442960353088 0 0 -1 7 0 0 0
[pid=4476] ppid=4363 vsize=578408 CPUtime=2.03 cores=0-7
/proc/4476/stat : 4476 (python) S 4363 4363 4156 0 -1 4202560 1238 0 0 0 108 95 0 0 18 0 2 0 660968717 592289792 28509 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4476/statm: 144602 28509 406 364 0 80478 0
[pid=4478] ppid=4476 vsize=191904 CPUtime=624.32 cores=0-7
/proc/4478/stat : 4478 (precosat) R 4476 4478 4478 0 -1 4202496 123495 0 0 0 62409 23 0 0 25 0 1 0 660968717 196509696 47378 33554432000 4194304 4890832 140736909512176 18446744073709551615 4297591 0 0 16781312 17442 0 0 0 17 2 0 0 0
/proc/4478/statm: 47976 47378 86 171 0 47803 0
[pid=4476/tid=4492] ppid=4363 vsize=578408 CPUtime=0 cores=0-7
/proc/4476/task/4492/stat : 4492 (python) S 4363 4363 4156 0 -1 4202560 95 0 0 0 0 0 0 0 19 0 2 0 661008237 592289792 28509 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604450817913088 0 0 -1 2 0 0 0
[pid=4493] ppid=4363 vsize=588664 CPUtime=0.78 cores=0-7
/proc/4493/stat : 4493 (python) S 4363 4363 4156 0 -1 4202560 961 0 0 0 42 36 0 0 18 0 1 0 661008387 602791936 28480 33554432000 4194304 5683644 140736271978032 18446744073709551615 254500712006 0 0 16781312 514 18446744073709519109 0 0 17 2 0 0 0
/proc/4493/statm: 147166 28480 379 364 0 83039 0
[pid=4495] ppid=4493 vsize=154900 CPUtime=233.93 cores=0-7
/proc/4495/stat : 4495 (precosat) R 4493 4495 4495 0 -1 4202496 96184 0 0 0 23379 14 0 0 25 0 1 0 661008387 158617600 38117 33554432000 4194304 4890832 140734246367936 18446744073709551615 4213100 0 0 16781312 17442 0 0 0 17 3 0 0 0
/proc/4495/statm: 38725 38117 86 171 0 38552 0
[pid=4500] ppid=4363 vsize=598920 CPUtime=0.34 cores=0-7
/proc/4500/stat : 4500 (python) S 4363 4363 4156 0 -1 4202560 1220 0 0 0 19 15 0 0 18 0 2 0 661019762 613294080 28469 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4500/statm: 149730 28469 406 364 0 85600 0
[pid=4502] ppid=4500 vsize=107680 CPUtime=98.68 cores=0-7
/proc/4502/stat : 4502 (precosat) T 4500 4502 4502 0 -1 4202496 40978 0 0 0 9861 7 0 0 25 0 1 0 661019762 110264320 26323 33554432000 4194304 4890832 140734791558176 18446744073709551615 4238265 0 0 16781312 17442 18446744071562691360 0 0 17 2 0 0 0
/proc/4502/statm: 26920 26323 86 171 0 26747 0
[pid=4500/tid=4505] ppid=4363 vsize=598920 CPUtime=0 cores=0-7
/proc/4500/task/4505/stat : 4505 (python) S 4363 4363 4156 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 661029795 613294080 28469 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 514 18446604447137599808 0 0 -1 2 0 0 0
[pid=4363/tid=4370] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4370/stat : 4370 (python) S 4361 4363 4156 0 -1 4202560 25 932 0 0 0 0 0 0 25 0 18 0 660812716 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 0 0 0 0
[pid=4363/tid=4373] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4373/stat : 4373 (python) S 4361 4363 4156 0 -1 4202560 34 932 0 0 0 0 0 0 18 0 18 0 660812719 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 0 0 0 0
[pid=4363/tid=4376] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4376/stat : 4376 (python) S 4361 4363 4156 0 -1 4202560 28 932 0 0 0 0 0 0 18 0 18 0 660812722 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 0 0 0 0
[pid=4363/tid=4379] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4379/stat : 4379 (python) S 4361 4363 4156 0 -1 4202560 29 932 0 0 0 0 0 0 18 0 18 0 660812724 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 6 0 0 0
[pid=4363/tid=4382] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4382/stat : 4382 (python) S 4361 4363 4156 0 -1 4202560 23 932 0 0 0 0 0 0 19 0 18 0 660812727 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 4 0 0 0
[pid=4363/tid=4385] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4385/stat : 4385 (python) S 4361 4363 4156 0 -1 4202560 20 932 0 0 0 0 0 0 19 0 18 0 660812730 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 7 0 0 0
[pid=4363/tid=4388] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4388/stat : 4388 (python) S 4361 4363 4156 0 -1 4202560 35 932 0 0 0 0 0 0 19 0 18 0 660812733 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 1 0 0 0
[pid=4363/tid=4391] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4391/stat : 4391 (python) S 4361 4363 4156 0 -1 4202560 43 932 0 0 0 0 0 0 18 0 18 0 660812736 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 2 0 0 0
[pid=4363/tid=4399] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4399/stat : 4399 (python) S 4361 4363 4156 0 -1 4202560 23 932 0 0 0 0 0 0 19 0 18 0 660822750 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 5 0 0 0
[pid=4363/tid=4402] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4402/stat : 4402 (python) S 4361 4363 4156 0 -1 4202560 30 932 0 0 0 0 0 0 20 0 18 0 660822754 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 1 0 0 0
[pid=4363/tid=4418] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4418/stat : 4418 (python) S 4361 4363 4156 0 -1 4202560 78 932 0 0 0 0 0 0 20 0 18 0 660861874 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 2 0 0 0
[pid=4363/tid=4425] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4425/stat : 4425 (python) S 4361 4363 4156 0 -1 4202560 42 932 0 0 0 0 0 0 18 0 18 0 660871812 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 1 0 0 0
[pid=4363/tid=4442] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4442/stat : 4442 (python) S 4361 4363 4156 0 -1 4202560 32 932 0 0 0 0 0 0 18 0 18 0 660920303 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 2 0 0 0
[pid=4363/tid=4468] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4468/stat : 4468 (python) S 4361 4363 4156 0 -1 4202560 48 932 0 0 0 0 0 0 18 0 18 0 660949434 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 1 0 0 0
[pid=4363/tid=4477] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4477/stat : 4477 (python) S 4361 4363 4156 0 -1 4202560 45 932 0 0 0 0 0 0 18 0 18 0 660968717 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 2 0 0 0
[pid=4363/tid=4494] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4494/stat : 4494 (python) S 4361 4363 4156 0 -1 4202560 40 932 0 0 0 0 0 0 20 0 18 0 661008387 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 1 0 0 0
[pid=4363/tid=4501] ppid=4361 vsize=609164 CPUtime=0 cores=0-7
/proc/4363/task/4501/stat : 4501 (python) S 4361 4363 4156 0 -1 4202560 28 932 0 0 0 0 0 0 20 0 18 0 661019762 623783936 30032 33554432000 4194304 5683644 140736271978032 18446744073709551615 254512515505 0 0 16781312 2 18446604450980581760 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 17497.3
Current children cumulated vsize (KiB) 31754132

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 4363 and gives
#  childrusage.ru_utime.tv_sec=24
#  childrusage.ru_utime.tv_usec=827225
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=227965
# CPU time returned by wait4() is 25.0552
# while last known CPU time is 17497.3
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 2216.12
CPU time (s): 17497.3
CPU user time (s): 17448.6
CPU system time (s): 48.67
CPU usage (%): 789.546
Max. virtual memory (cumulated for all children) (KiB): 31754132

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 24.8272
system time used= 0.227965
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 60471
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3066
involuntary context switches= 124

runsolver used 17.4663 second user time and 40.3239 second system time

The end

Launcher Data 

Begin job on node149 at 2011-05-15 22:42:16
IDJOB=3380784
IDBENCH=82381
IDSOLVER=1864
FILE ID=node149/3380784-1305492135
RUNJOBID= node149-1305487124-4170
PBS_JOBID= 13325384
Free space on /tmp= 73544 MiB

SOLVER NAME= borg-sat 11.03.02-par-industrial (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.025-NOTKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380784-1305492135/watcher-3380784-1305492135 -o /tmp/evaluation-result-3380784-1305492135/solver-3380784-1305492135 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3380784-1305492135.cnf -seed 1302967919 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 3c5c73741ba21d5f39c7f436ba834743
RANDOM SEED=1302967919

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      22983476 kB
Buffers:        334128 kB
Cached:        4567044 kB
SwapCached:      19976 kB
Active:        7046128 kB
Inactive:      2607072 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      22983476 kB
SwapTotal:    67111528 kB
SwapFree:     45937200 kB
Dirty:           39980 kB
Writeback:           0 kB
AnonPages:     4745156 kB
Mapped:          21952 kB
Slab:           166456 kB
PageTables:      79544 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 120764012 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73512 MiB
End job on node149 at 2011-05-15 23:19:14