Trace number 3317928

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-industrial? (TO) 4800.39 1221.67

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_1.025-NOTKNOWN.cnf
MD5SUM732b9c9b0d44d6a9e75489b2f5004165
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables95456
Number of clauses477186
Sum of the clauses size1137161
Maximum clause length63
Minimum clause length1
Number of clauses of size 1211
Number of clauses of size 2470925
Number of clauses of size 31500
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 54550

Solver Data 

0.32/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.32/0.67	c INFO: solving HOME/instance-3317928-1304268750.cnf
0.32/0.67	c INFO: computing task features
1.70/2.08	c INFO: computed variable-clause graph statistics
1.81/2.14	c INFO: computed clause balance statistics
8.60/8.93	c INFO: computed variable balance statistics
13.40/13.73	c INFO: computed small-clause counts
13.40/13.80	c INFO: computed Horn-clause counts
24.30/24.63	c INFO: computed variable graph statistics
24.30/24.63	c INFO: collected features for HOME/instance-3317928-1304268750.cnf in 23.91 s
24.30/24.65	c INFO: running SApperloT@0 for 97 with 4776 remaining (b = 0.13)
24.30/24.68	c INFO: running lingeling-276@0 for 97 with 4776 remaining (b = 0.15)
24.40/24.71	c INFO: running cryptominisat-2.9.0@0 for 485 with 4775 remaining (b = 0.22)
24.40/24.73	c INFO: running lingeling-276@0 for 1261 with 4775 remaining (b = 0.37)
420.11/123.82	c INFO: running lingeling-276@0 for 1067 with 4676 remaining (b = 0.27)
424.09/124.84	c INFO: running precosat-570@0 for 291 with 4675 remaining (b = 0.10)
1591.79/417.24	c INFO: running cryptominisat-2.9.0@0 for 485 with 4383 remaining (b = 0.14)
1977.11/513.75	c INFO: running lingeling-276@0 for 582 with 4286 remaining (b = 0.13)
3556.52/909.42	c INFO: running lingeling-276@0 for 97 with 3891 remaining (b = 0.06)
3955.33/1009.59	c INFO: running precosat-570@291 for 97 with 3791 remaining (b = 0.03)
4305.94/1097.54	c INFO: running cryptominisat-2.9.0@486 for 1455 with 3703 remaining (b = 0.36)
4362.54/1111.78	c INFO: running cryptominisat-2.9.0@0 for 1843 with 3688 remaining (b = 0.36)
4699.23/1196.30	c INFO: running cryptominisat-2.9.0@0 for 97 with 3604 remaining (b = 0.03)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317928-1304268750/watcher-3317928-1304268750 -o /tmp/evaluation-result-3317928-1304268750/solver-3317928-1304268750 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317928-1304268750.cnf -seed 1512910626 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.26 6.99 5.93 5/205 5592
/proc/meminfo: memFree=29532084/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=15384 CPUtime=0 cores=0,2,4,6
/proc/5592/stat : 5592 (python) R 5590 5592 5278 0 -1 4202496 795 0 0 0 0 0 0 0 24 0 1 0 538474882 15753216 681 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089696256 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/5592/statm: 3846 681 322 364 0 333 0

[startup+0.0859901 s]
/proc/loadavg: 6.26 6.99 5.93 5/205 5592
/proc/meminfo: memFree=29532084/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=54540 CPUtime=0.04 cores=0,2,4,6
/proc/5592/stat : 5592 (python) S 5590 5592 5278 0 -1 4202496 2057 0 0 0 3 1 0 0 24 0 1 0 538474882 55848960 1814 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089696256 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/5592/statm: 13635 1814 492 364 0 1285 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 54540

[startup+0.100966 s]
/proc/loadavg: 6.26 6.99 5.93 5/205 5592
/proc/meminfo: memFree=29532084/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=89772 CPUtime=0.06 cores=0,2,4,6
/proc/5592/stat : 5592 (python) S 5590 5592 5278 0 -1 4202496 2673 0 0 0 4 2 0 0 24 0 1 0 538474882 91926528 2388 33554432000 4194304 5683644 140735875111200 18446744073709551615 248084774535 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/5592/statm: 22443 2388 750 364 0 1623 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 89772

[startup+0.300949 s]
/proc/loadavg: 6.26 6.99 5.93 5/205 5592
/proc/meminfo: memFree=29532084/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=135752 CPUtime=0.18 cores=0,2,4,6
/proc/5592/stat : 5592 (python) R 5590 5592 5278 0 -1 4202496 5963 935 1 0 12 6 0 0 25 0 1 0 538474882 139010048 4150 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089694213 0 2147483391 16781312 2 0 0 0 17 6 0 0 0
/proc/5592/statm: 33938 4150 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 135752

[startup+0.700852 s]
/proc/loadavg: 6.26 6.99 5.93 5/205 5592
/proc/meminfo: memFree=29532084/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=287632 CPUtime=0.41 cores=0,2,4,6
/proc/5592/stat : 5592 (python) R 5590 5592 5278 0 -1 4202496 11551 935 1 0 27 14 0 0 25 0 1 0 538474882 294535168 9231 33554432000 4194304 5683644 140735875111200 18446744073709551615 4539767 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/5592/statm: 71908 9231 1761 364 0 7832 0
Current children cumulated CPU time (s) 0.41
Current children cumulated vsize (KiB) 287632

[startup+1.50071 s]
/proc/loadavg: 6.26 6.99 5.93 6/206 5597
/proc/meminfo: memFree=29480740/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=341052 CPUtime=1.2 cores=0,2,4,6
/proc/5592/stat : 5592 (python) R 5590 5592 5278 0 -1 4202496 24672 935 1 0 104 16 0 0 25 0 1 0 538474882 349237248 22352 33554432000 4194304 5683644 140735875111200 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/5592/statm: 85263 22352 1761 364 0 21187 0
Current children cumulated CPU time (s) 1.2
Current children cumulated vsize (KiB) 341052

[startup+3.10042 s]
/proc/loadavg: 6.26 6.99 5.93 6/206 5597
/proc/meminfo: memFree=29410928/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=375628 CPUtime=2.8 cores=0,2,4,6
/proc/5592/stat : 5592 (python) R 5590 5592 5278 0 -1 4202496 44453 935 1 0 262 18 0 0 25 0 1 0 538474882 384643072 31291 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101535415 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/5592/statm: 93907 31291 1826 364 0 29831 0
Current children cumulated CPU time (s) 2.8
Current children cumulated vsize (KiB) 375628

[startup+6.30086 s]
/proc/loadavg: 6.16 6.95 5.92 6/206 5597
/proc/meminfo: memFree=29408200/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=375628 CPUtime=6 cores=0,2,4,6
/proc/5592/stat : 5592 (python) R 5590 5592 5278 0 -1 4202496 44453 935 1 0 582 18 0 0 25 0 1 0 538474882 384643072 31291 33554432000 4194304 5683644 140735875111200 18446744073709551615 4272072 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/5592/statm: 93907 31291 1826 364 0 29831 0
Current children cumulated CPU time (s) 6
Current children cumulated vsize (KiB) 375628

[startup+12.7007 s]
/proc/loadavg: 6.07 6.92 5.92 6/206 5597
/proc/meminfo: memFree=29411024/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=375628 CPUtime=12.4 cores=0,2,4,6
/proc/5592/stat : 5592 (python) R 5590 5592 5278 0 -1 4202496 44454 935 1 0 1222 18 0 0 25 0 1 0 538474882 384643072 31292 33554432000 4194304 5683644 140735875111200 18446744073709551615 47969349036856 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/5592/statm: 93907 31292 1827 364 0 29831 0
Current children cumulated CPU time (s) 12.4
Current children cumulated vsize (KiB) 375628

[startup+25.5015 s]
/proc/loadavg: 5.83 6.82 5.90 9/218 5610
/proc/meminfo: memFree=29337516/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=409240 CPUtime=24.44 cores=0,2,4,6
/proc/5592/stat : 5592 (python) S 5590 5592 5278 0 -1 4202496 47116 935 1 0 2424 20 0 0 25 0 5 0 538474882 419061760 29506 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/5592/statm: 102310 29506 1881 364 0 38219 0
[pid=5599] ppid=5592 vsize=368228 CPUtime=0 cores=0,2,4,6
/proc/5599/stat : 5599 (python) S 5592 5592 5278 0 -1 4202560 791 0 0 0 0 0 0 0 25 0 1 0 538477348 377065472 27960 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718287 0 0 16781312 514 4011020346567163903 0 0 17 6 0 0 0
/proc/5599/statm: 92057 27960 351 364 0 27975 0
[pid=5601] ppid=5599 vsize=16120 CPUtime=0.83 cores=0,2,4,6
/proc/5601/stat : 5601 (SApperloT-base) R 5599 5601 5601 0 -1 4202496 3604 0 0 0 82 1 0 0 25 0 1 0 538477348 16506880 3308 33554432000 134512640 135674940 4294939200 18446744073709551615 134558972 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/5601/statm: 4030 3308 99 284 0 3744 0
[pid=5602] ppid=5592 vsize=378484 CPUtime=0 cores=0,2,4,6
/proc/5602/stat : 5602 (python) S 5592 5592 5278 0 -1 4202560 727 0 0 0 0 0 0 0 25 0 1 0 538477350 387567616 27956 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 9461424280022548479 0 0 17 4 0 0 0
/proc/5602/statm: 94621 27956 343 364 0 30536 0
[pid=5604] ppid=5602 vsize=13984 CPUtime=0.79 cores=0,2,4,6
/proc/5604/stat : 5604 (lingeling) R 5602 5604 5604 0 -1 4202496 6695 0 0 0 78 1 0 0 25 0 1 0 538477351 14319616 2637 33554432000 4194304 4352236 140733487693280 18446744073709551615 4258983 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/5604/statm: 3496 2637 118 39 0 2576 0
[pid=5605] ppid=5592 vsize=388740 CPUtime=0 cores=0,2,4,6
/proc/5605/stat : 5605 (python) S 5592 5592 5278 0 -1 4202560 750 0 0 0 0 0 0 0 21 0 1 0 538477353 398069760 27964 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 3754691475661619460 0 0 17 4 0 0 0
/proc/5605/statm: 97185 27964 347 364 0 33097 0
[pid=5607] ppid=5605 vsize=95276 CPUtime=0.77 cores=0,2,4,6
/proc/5607/stat : 5607 (cryptominisat-2) R 5605 5607 5607 0 -1 4202496 17475 0 0 0 74 3 0 0 25 0 1 0 538477354 97562624 15017 33554432000 4194304 4677096 140734374004976 18446744073709551615 4236529 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/5607/statm: 23819 15017 361 118 0 18778 0
[pid=5608] ppid=5592 vsize=398996 CPUtime=0 cores=0,2,4,6
/proc/5608/stat : 5608 (python) S 5592 5592 5278 0 -1 4202560 740 0 0 0 0 0 0 0 25 0 1 0 538477356 408571904 27964 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 9461416583441154047 0 0 17 2 0 0 0
/proc/5608/statm: 99749 27964 343 364 0 35658 0
[pid=5610] ppid=5608 vsize=13960 CPUtime=0.74 cores=0,2,4,6
/proc/5610/stat : 5610 (lingeling) R 5608 5610 5610 0 -1 4202496 4917 0 0 0 74 0 0 0 25 0 1 0 538477356 14295040 2643 33554432000 4194304 4352236 140735923645776 18446744073709551615 4262885 0 0 16781312 17506 0 0 0 17 2 0 0 0
/proc/5610/statm: 3490 2643 118 39 0 2570 0
[pid=5592/tid=5600] ppid=5590 vsize=409240 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5600/stat : 5600 (python) S 5590 5592 5278 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 5 0 538477348 419061760 29506 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5603] ppid=5590 vsize=409240 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5603/stat : 5603 (python) S 5590 5592 5278 0 -1 4202560 22 935 0 0 0 0 0 0 25 0 5 0 538477351 419061760 29506 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5606] ppid=5590 vsize=409240 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5606/stat : 5606 (python) S 5590 5592 5278 0 -1 4202560 23 935 0 0 0 0 0 0 25 0 5 0 538477354 419061760 29506 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5609] ppid=5590 vsize=409240 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5609/stat : 5609 (python) S 5590 5592 5278 0 -1 4202560 28 935 0 0 0 0 0 0 25 0 5 0 538477356 419061760 29506 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 27.57
Current children cumulated vsize (KiB) 2083028

[startup+51.1009 s]
/proc/loadavg: 6.57 6.92 5.96 9/218 5610
/proc/meminfo: memFree=28678244/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=409240 CPUtime=24.44 cores=0,2,4,6
/proc/5592/stat : 5592 (python) S 5590 5592 5278 0 -1 4202496 47116 935 1 0 2424 20 0 0 25 0 5 0 538474882 419061760 29506 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/5592/statm: 102310 29506 1881 364 0 38219 0
[pid=5599] ppid=5592 vsize=368228 CPUtime=0.02 cores=0,2,4,6
/proc/5599/stat : 5599 (python) S 5592 5592 5278 0 -1 4202560 1002 0 0 0 1 1 0 0 18 0 1 0 538477348 377065472 27990 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718287 0 0 16781312 514 13296578832648601860 0 0 17 6 0 0 0
/proc/5599/statm: 92057 27990 381 364 0 27975 0
[pid=5601] ppid=5599 vsize=38572 CPUtime=26.41 cores=0,2,4,6
/proc/5601/stat : 5601 (SApperloT-base) R 5599 5601 5601 0 -1 4202496 9298 0 0 0 2640 1 0 0 25 0 1 0 538477348 39497728 8808 33554432000 134512640 135674940 4294939200 18446744073709551615 134551972 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/5601/statm: 9643 8808 99 284 0 9357 0
[pid=5602] ppid=5592 vsize=378484 CPUtime=0.02 cores=0,2,4,6
/proc/5602/stat : 5602 (python) S 5592 5592 5278 0 -1 4202560 922 0 0 0 1 1 0 0 18 0 1 0 538477350 387567616 27983 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 13296578832648601860 0 0 17 4 0 0 0
/proc/5602/statm: 94621 27983 370 364 0 30536 0
[pid=5604] ppid=5602 vsize=41276 CPUtime=26.19 cores=0,2,4,6
/proc/5604/stat : 5604 (lingeling) R 5602 5604 5604 0 -1 4202496 13669 0 0 0 2618 1 0 0 25 0 1 0 538477351 42266624 9490 33554432000 4194304 4352236 140733487693280 18446744073709551615 4231316 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/5604/statm: 10319 9490 129 39 0 9399 0
[pid=5605] ppid=5592 vsize=388740 CPUtime=0.07 cores=0,2,4,6
/proc/5605/stat : 5605 (python) S 5592 5592 5278 0 -1 4202560 1022 0 0 0 5 2 0 0 15 0 1 0 538477353 398069760 27988 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 3754691475661619460 0 0 17 4 0 0 0
/proc/5605/statm: 97185 27988 371 364 0 33097 0
[pid=5607] ppid=5605 vsize=839800 CPUtime=26.26 cores=0,2,4,6
/proc/5607/stat : 5607 (cryptominisat-2) R 5605 5607 5607 0 -1 4202496 193205 0 0 0 2601 25 0 0 25 0 1 0 538477354 859955200 160717 33554432000 4194304 4677096 140734374004976 18446744073709551615 4365099 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/5607/statm: 209950 160717 390 118 0 204909 0
[pid=5608] ppid=5592 vsize=398996 CPUtime=0.02 cores=0,2,4,6
/proc/5608/stat : 5608 (python) S 5592 5592 5278 0 -1 4202560 941 0 0 0 1 1 0 0 18 0 1 0 538477356 408571904 27991 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 13296578832648601860 0 0 17 2 0 0 0
/proc/5608/statm: 99749 27991 370 364 0 35658 0
[pid=5610] ppid=5608 vsize=35768 CPUtime=26.31 cores=0,2,4,6
/proc/5610/stat : 5610 (lingeling) R 5608 5610 5610 0 -1 4202496 12291 0 0 0 2630 1 0 0 25 0 1 0 538477356 36626432 8112 33554432000 4194304 4352236 140735923645776 18446744073709551615 4276514 0 0 16781312 17506 0 0 0 17 2 0 0 0
/proc/5610/statm: 8942 8112 129 39 0 8022 0
[pid=5592/tid=5600] ppid=5590 vsize=409240 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5600/stat : 5600 (python) S 5590 5592 5278 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 5 0 538477348 419061760 29506 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5603] ppid=5590 vsize=409240 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5603/stat : 5603 (python) S 5590 5592 5278 0 -1 4202560 22 935 0 0 0 0 0 0 25 0 5 0 538477351 419061760 29506 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5606] ppid=5590 vsize=409240 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5606/stat : 5606 (python) S 5590 5592 5278 0 -1 4202560 23 935 0 0 0 0 0 0 25 0 5 0 538477354 419061760 29506 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5609] ppid=5590 vsize=409240 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5609/stat : 5609 (python) S 5590 5592 5278 0 -1 4202560 28 935 0 0 0 0 0 0 25 0 5 0 538477356 419061760 29506 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 129.74
Current children cumulated vsize (KiB) 2899104

[startup+102.308 s]
/proc/loadavg: 7.38 7.08 6.06 9/218 5612
/proc/meminfo: memFree=25864720/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=409240 CPUtime=24.44 cores=0,2,4,6
/proc/5592/stat : 5592 (python) S 5590 5592 5278 0 -1 4202496 47116 935 1 0 2424 20 0 0 25 0 5 0 538474882 419061760 29506 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/5592/statm: 102310 29506 1881 364 0 38219 0
[pid=5599] ppid=5592 vsize=368228 CPUtime=0.1 cores=0,2,4,6
/proc/5599/stat : 5599 (python) S 5592 5592 5278 0 -1 4202560 1025 0 0 0 6 4 0 0 18 0 1 0 538477348 377065472 27990 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718287 0 0 16781312 514 13296578832648601860 0 0 17 6 0 0 0
/proc/5599/statm: 92057 27990 381 364 0 27975 0
[pid=5601] ppid=5599 vsize=80152 CPUtime=77.62 cores=0,2,4,6
/proc/5601/stat : 5601 (SApperloT-base) R 5599 5601 5601 0 -1 4202496 19701 0 0 0 7757 5 0 0 25 0 1 0 538477348 82075648 19211 33554432000 134512640 135674940 4294939200 18446744073709551615 134552032 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/5601/statm: 20038 19211 99 284 0 19752 0
[pid=5602] ppid=5592 vsize=378484 CPUtime=0.1 cores=0,2,4,6
/proc/5602/stat : 5602 (python) S 5592 5592 5278 0 -1 4202560 942 0 0 0 6 4 0 0 18 0 1 0 538477350 387567616 27983 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 13296578832648601860 0 0 17 4 0 0 0
/proc/5602/statm: 94621 27983 370 364 0 30536 0
[pid=5604] ppid=5602 vsize=69916 CPUtime=77 cores=0,2,4,6
/proc/5604/stat : 5604 (lingeling) R 5602 5604 5604 0 -1 4202496 20916 0 0 0 7698 2 0 0 25 0 1 0 538477351 71593984 16650 33554432000 4194304 4352236 140733487693280 18446744073709551615 4281284 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/5604/statm: 17479 16650 129 39 0 16559 0

################
# More data... #
################

/proc/5608/statm: 99749 27991 370 364 0 35658 0
[pid=5610] ppid=5608 vsize=183100 CPUtime=1148.62 cores=0,2,4,6
/proc/5610/stat : 5610 (lingeling) R 5608 5610 5610 0 -1 4202496 325802 0 0 0 114821 41 0 0 25 0 1 0 538477356 187494400 44945 33554432000 4194304 4352236 140735923645776 18446744073709551615 4329086 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/5610/statm: 45775 44945 129 39 0 44855 0
[pid=5614] ppid=5592 vsize=409252 CPUtime=1.54 cores=0,2,4,6
/proc/5614/stat : 5614 (python) S 5592 5592 5278 0 -1 4202560 1006 0 0 0 84 70 0 0 18 0 1 0 538487264 419074048 27995 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 13296578832648601860 0 0 17 2 0 0 0
/proc/5614/statm: 102313 27995 370 364 0 38219 0
[pid=5616] ppid=5614 vsize=126872 CPUtime=1056.19 cores=0,2,4,6
/proc/5616/stat : 5616 (lingeling) R 5614 5616 5616 0 -1 4202496 376461 0 0 0 105576 43 0 0 25 0 1 0 538487265 129916928 30921 33554432000 4194304 4352236 140734482636608 18446744073709551615 4328299 0 0 16781312 17506 0 0 0 17 2 0 0 0
/proc/5616/statm: 31718 30921 129 39 0 30798 0
[pid=5618] ppid=5592 vsize=419508 CPUtime=0.57 cores=0,2,4,6
/proc/5618/stat : 5618 (python) S 5592 5592 5278 0 -1 4202560 1162 0 0 0 29 28 0 0 18 0 2 0 538487366 429576192 28029 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/5618/statm: 104877 28029 399 364 0 40780 0
[pid=5620] ppid=5618 vsize=158008 CPUtime=394.02 cores=0,2,4,6
/proc/5620/stat : 5620 (precosat) T 5618 5620 5620 0 -1 4202496 116984 0 0 0 39388 14 0 0 25 0 1 0 538487367 161800192 38891 33554432000 4194304 4890832 140736900387040 18446744073709551615 4238686 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/5620/statm: 39502 38891 86 171 0 39329 0
[pid=5618/tid=5635] ppid=5592 vsize=419508 CPUtime=0 cores=0,2,4,6
/proc/5618/task/5635/stat : 5635 (python) S 5592 5592 5278 0 -1 4202560 45 0 0 0 0 0 0 0 18 0 2 0 538516604 429576192 28029 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 514 18446604453865686976 0 0 -1 4 0 0 0
[pid=5636] ppid=5592 vsize=429764 CPUtime=1.56 cores=0,2,4,6
/proc/5636/stat : 5636 (python) S 5592 5592 5278 0 -1 4202560 2201 0 0 0 103 53 0 0 15 0 2 0 538516606 440078336 28034 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/5636/statm: 107441 28034 400 364 0 43341 0
[pid=5638] ppid=5636 vsize=3992348 CPUtime=573.79 cores=0,2,4,6
/proc/5638/stat : 5638 (cryptominisat-2) R 5636 5638 5638 0 -1 4202496 1253407 0 0 0 57197 182 0 0 25 0 1 0 538516607 4088164352 927664 33554432000 4194304 4677096 140733973184272 18446744073709551615 4317855 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/5638/statm: 998087 927664 397 118 0 993046 0
[pid=5636/tid=5670] ppid=5592 vsize=429764 CPUtime=0 cores=0,2,4,6
/proc/5636/task/5670/stat : 5670 (python) S 5592 5592 5278 0 -1 4202560 48 0 0 0 0 0 0 0 17 0 2 0 538565822 440078336 28034 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 514 18446604446969385408 0 0 -1 2 0 0 0
[pid=5671] ppid=5592 vsize=440020 CPUtime=0.14 cores=0,2,4,6
/proc/5671/stat : 5671 (python) S 5592 5592 5278 0 -1 4202560 1175 0 0 0 7 7 0 0 19 0 2 0 538565824 450580480 28043 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/5671/statm: 110005 28043 405 364 0 45902 0
[pid=5673] ppid=5671 vsize=72264 CPUtime=98.41 cores=0,2,4,6
/proc/5673/stat : 5673 (lingeling) T 5671 5673 5673 0 -1 4202496 21427 0 0 0 9837 4 0 0 25 0 1 0 538565825 73998336 17235 33554432000 4194304 4352236 140737139675072 18446744073709551615 4278721 0 0 16781312 17506 18446744071562691360 0 0 17 6 0 0 0
/proc/5673/statm: 18066 17235 129 39 0 17146 0
[pid=5671/tid=5713] ppid=5592 vsize=440020 CPUtime=0 cores=0,2,4,6
/proc/5671/task/5713/stat : 5713 (python) S 5592 5592 5278 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538575839 450580480 28043 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 514 18446604453846936256 0 0 -1 6 0 0 0
[pid=5592/tid=5600] ppid=5590 vsize=450264 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5600/stat : 5600 (python) S 5590 5592 5278 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 9 0 538477348 461070336 29531 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5603] ppid=5590 vsize=450264 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5603/stat : 5603 (python) S 5590 5592 5278 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 9 0 538477351 461070336 29531 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5606] ppid=5590 vsize=450264 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5606/stat : 5606 (python) S 5590 5592 5278 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 9 0 538477354 461070336 29531 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5609] ppid=5590 vsize=450264 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5609/stat : 5609 (python) S 5590 5592 5278 0 -1 4202560 28 935 0 0 0 0 0 0 25 0 9 0 538477356 461070336 29531 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
[pid=5592/tid=5615] ppid=5590 vsize=450264 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5615/stat : 5615 (python) S 5590 5592 5278 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 9 0 538487265 461070336 29531 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
[pid=5592/tid=5619] ppid=5590 vsize=450264 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5619/stat : 5619 (python) S 5590 5592 5278 0 -1 4202560 34 935 0 0 0 0 0 0 18 0 9 0 538487367 461070336 29531 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
[pid=5592/tid=5637] ppid=5590 vsize=450264 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5637/stat : 5637 (python) S 5590 5592 5278 0 -1 4202560 30 935 0 0 0 0 0 0 18 0 9 0 538516607 461070336 29531 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
[pid=5592/tid=5672] ppid=5590 vsize=450264 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5672/stat : 5672 (python) S 5590 5592 5278 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 9 0 538565825 461070336 29531 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4643.83
Current children cumulated vsize (KiB) 12458736



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1221.6 s]
/proc/loadavg: 9.13 8.41 7.59 10/241 5739
/proc/meminfo: memFree=20001808/32950928 swapFree=67111428/67111528
[pid=5592] ppid=5590 vsize=460520 CPUtime=24.68 cores=0,2,4,6
/proc/5592/stat : 5592 (python) S 5590 5592 5278 0 -1 4202496 51306 935 1 0 2445 23 0 0 19 0 10 0 538474882 471572480 29537 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/5592/statm: 115130 29537 1892 364 0 51024 0
[pid=5599] ppid=5592 vsize=378472 CPUtime=0.14 cores=0,2,4,6
/proc/5599/stat : 5599 (python) S 5592 5592 5278 0 -1 4202560 1190 0 0 0 8 6 0 0 19 0 2 0 538477348 387555328 28028 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/5599/statm: 94618 28028 414 364 0 30536 0
[pid=5601] ppid=5599 vsize=101236 CPUtime=99.11 cores=0,2,4,6
/proc/5601/stat : 5601 (SApperloT-base) T 5599 5601 5601 0 -1 4202496 24963 0 0 0 9904 7 0 0 25 0 1 0 538477348 103665664 24473 33554432000 134512640 135674940 4294939200 18446744073709551615 134564617 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/5601/statm: 25309 24473 99 284 0 25023 0
[pid=5599/tid=5613] ppid=5592 vsize=378472 CPUtime=0 cores=0,2,4,6
/proc/5599/task/5613/stat : 5613 (python) S 5592 5592 5278 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538487262 387555328 28028 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 514 18446604453846581696 0 0 -1 6 0 0 0
[pid=5602] ppid=5592 vsize=378484 CPUtime=1 cores=0,2,4,6
/proc/5602/stat : 5602 (python) S 5592 5592 5278 0 -1 4202560 1156 0 0 0 54 46 0 0 18 0 2 0 538477350 387567616 28013 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/5602/statm: 94621 28013 399 364 0 30536 0
[pid=5604] ppid=5602 vsize=97480 CPUtime=682.23 cores=0,2,4,6
/proc/5604/stat : 5604 (lingeling) T 5602 5604 5604 0 -1 4202496 196437 0 0 0 68198 25 0 0 25 0 1 0 538477351 99819520 23573 33554432000 4194304 4352236 140733487693280 18446744073709551615 4328109 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/5604/statm: 24370 23573 129 39 0 23450 0
[pid=5602/tid=5617] ppid=5592 vsize=378484 CPUtime=0 cores=0,2,4,6
/proc/5602/task/5617/stat : 5617 (python) S 5592 5592 5278 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 538487364 387567616 28013 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 514 18446604446525668160 0 0 -1 4 0 0 0
[pid=5605] ppid=5592 vsize=388740 CPUtime=1.54 cores=0,2,4,6
/proc/5605/stat : 5605 (python) S 5592 5592 5278 0 -1 4202560 2169 0 0 0 101 53 0 0 15 0 2 0 538477353 398069760 28018 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/5605/statm: 97185 28018 400 364 0 33097 0
[pid=5607] ppid=5605 vsize=3973212 CPUtime=598.05 cores=0,2,4,6
/proc/5607/stat : 5607 (cryptominisat-2) R 5605 5607 5607 0 -1 4202496 1263791 0 0 0 59619 186 0 0 25 0 1 0 538477354 4068569088 937795 33554432000 4194304 4677096 140734374004976 18446744073709551615 4319729 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/5607/statm: 993303 937795 394 118 0 988262 0
[pid=5605/tid=5651] ppid=5592 vsize=388740 CPUtime=0 cores=0,2,4,6
/proc/5605/task/5651/stat : 5651 (python) S 5592 5592 5278 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 538526255 398069760 28018 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 514 18446604446525667264 0 0 -1 4 0 0 0
[pid=5608] ppid=5592 vsize=398996 CPUtime=1.72 cores=0,2,4,6
/proc/5608/stat : 5608 (python) S 5592 5592 5278 0 -1 4202560 1005 0 0 0 93 79 0 0 18 0 1 0 538477356 408571904 27991 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 13296578832648601860 0 0 17 2 0 0 0
/proc/5608/statm: 99749 27991 370 364 0 35658 0
[pid=5610] ppid=5608 vsize=146232 CPUtime=1187.28 cores=0,2,4,6
/proc/5610/stat : 5610 (lingeling) R 5608 5610 5610 0 -1 4202496 341162 0 0 0 118685 43 0 0 25 0 1 0 538477356 149741568 35728 33554432000 4194304 4352236 140735923645776 18446744073709551615 4278683 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/5610/statm: 36558 35728 129 39 0 35638 0
[pid=5614] ppid=5592 vsize=409252 CPUtime=1.56 cores=0,2,4,6
/proc/5614/stat : 5614 (python) S 5592 5592 5278 0 -1 4202560 1160 0 0 0 85 71 0 0 19 0 2 0 538487264 419074048 28025 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/5614/statm: 102313 28025 399 364 0 38219 0
[pid=5616] ppid=5614 vsize=130968 CPUtime=1070.1 cores=0,2,4,6
/proc/5616/stat : 5616 (lingeling) T 5614 5616 5616 0 -1 4202496 377485 0 0 0 106967 43 0 0 25 0 1 0 538487265 134111232 31945 33554432000 4194304 4352236 140734482636608 18446744073709551615 4328100 0 0 16781312 17506 18446744071562691360 0 0 17 2 0 0 0
/proc/5616/statm: 32742 31945 129 39 0 31822 0
[pid=5614/tid=5735] ppid=5592 vsize=409252 CPUtime=0 cores=0,2,4,6
/proc/5614/task/5735/stat : 5735 (python) S 5592 5592 5278 0 -1 4202560 44 0 0 0 0 0 0 0 19 0 2 0 538594510 419074048 28025 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 514 18446604453870414784 0 0 -1 2 0 0 0
[pid=5618] ppid=5592 vsize=419508 CPUtime=0.57 cores=0,2,4,6
/proc/5618/stat : 5618 (python) S 5592 5592 5278 0 -1 4202560 1162 0 0 0 29 28 0 0 18 0 2 0 538487366 429576192 28029 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/5618/statm: 104877 28029 399 364 0 40780 0
[pid=5620] ppid=5618 vsize=158008 CPUtime=394.02 cores=0,2,4,6
/proc/5620/stat : 5620 (precosat) T 5618 5620 5620 0 -1 4202496 116984 0 0 0 39388 14 0 0 25 0 1 0 538487367 161800192 38891 33554432000 4194304 4890832 140736900387040 18446744073709551615 4238686 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/5620/statm: 39502 38891 86 171 0 39329 0
[pid=5618/tid=5635] ppid=5592 vsize=419508 CPUtime=0 cores=0,2,4,6
/proc/5618/task/5635/stat : 5635 (python) S 5592 5592 5278 0 -1 4202560 45 0 0 0 0 0 0 0 18 0 2 0 538516604 429576192 28029 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 514 18446604453865686976 0 0 -1 4 0 0 0
[pid=5636] ppid=5592 vsize=429764 CPUtime=1.62 cores=0,2,4,6
/proc/5636/stat : 5636 (python) S 5592 5592 5278 0 -1 4202560 2201 0 0 0 106 56 0 0 15 0 2 0 538516606 440078336 28034 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/5636/statm: 107441 28034 400 364 0 43341 0
[pid=5638] ppid=5636 vsize=3994696 CPUtime=612.95 cores=0,2,4,6
/proc/5638/stat : 5638 (cryptominisat-2) R 5636 5638 5638 0 -1 4202496 1273977 0 0 0 61109 186 0 0 25 0 1 0 538516607 4090568704 926948 33554432000 4194304 4677096 140733973184272 18446744073709551615 248089394836 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/5638/statm: 998674 926948 397 118 0 993633 0
[pid=5636/tid=5670] ppid=5592 vsize=429764 CPUtime=0 cores=0,2,4,6
/proc/5636/task/5670/stat : 5670 (python) S 5592 5592 5278 0 -1 4202560 48 0 0 0 0 0 0 0 17 0 2 0 538565822 440078336 28034 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 514 18446604446969385408 0 0 -1 2 0 0 0
[pid=5671] ppid=5592 vsize=440020 CPUtime=0.14 cores=0,2,4,6
/proc/5671/stat : 5671 (python) S 5592 5592 5278 0 -1 4202560 1175 0 0 0 7 7 0 0 19 0 2 0 538565824 450580480 28043 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/5671/statm: 110005 28043 405 364 0 45902 0
[pid=5673] ppid=5671 vsize=72264 CPUtime=98.41 cores=0,2,4,6
/proc/5673/stat : 5673 (lingeling) T 5671 5673 5673 0 -1 4202496 21427 0 0 0 9837 4 0 0 25 0 1 0 538565825 73998336 17235 33554432000 4194304 4352236 140737139675072 18446744073709551615 4278721 0 0 16781312 17506 18446744071562691360 0 0 17 6 0 0 0
/proc/5673/statm: 18066 17235 129 39 0 17146 0
[pid=5671/tid=5713] ppid=5592 vsize=440020 CPUtime=0 cores=0,2,4,6
/proc/5671/task/5713/stat : 5713 (python) S 5592 5592 5278 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538575839 450580480 28043 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 514 18446604453846936256 0 0 -1 6 0 0 0
[pid=5736] ppid=5592 vsize=450276 CPUtime=0.03 cores=0,2,4,6
/proc/5736/stat : 5736 (python) S 5592 5592 5278 0 -1 4202560 976 0 0 0 1 2 0 0 15 0 1 0 538594512 461082624 28011 33554432000 4194304 5683644 140735875111200 18446744073709551615 248089718342 0 0 16781312 514 4124197751338533124 0 0 17 2 0 0 0
/proc/5736/statm: 112569 28011 370 364 0 48463 0
[pid=5738] ppid=5736 vsize=737808 CPUtime=25.24 cores=0,2,4,6
/proc/5738/stat : 5738 (cryptominisat-2) R 5736 5738 5738 0 -1 4202496 143407 0 0 0 2506 18 0 0 25 0 1 0 538594513 755515392 134488 33554432000 4194304 4677096 140736737536640 18446744073709551615 4336461 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/5738/statm: 184452 134488 387 118 0 179411 0
[pid=5592/tid=5600] ppid=5590 vsize=460520 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5600/stat : 5600 (python) S 5590 5592 5278 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 10 0 538477348 471572480 29537 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5603] ppid=5590 vsize=460520 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5603/stat : 5603 (python) S 5590 5592 5278 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 10 0 538477351 471572480 29537 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5606] ppid=5590 vsize=460520 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5606/stat : 5606 (python) S 5590 5592 5278 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 10 0 538477354 471572480 29537 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 4 0 0 0
[pid=5592/tid=5609] ppid=5590 vsize=460520 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5609/stat : 5609 (python) S 5590 5592 5278 0 -1 4202560 28 935 0 0 0 0 0 0 25 0 10 0 538477356 471572480 29537 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
[pid=5592/tid=5615] ppid=5590 vsize=460520 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5615/stat : 5615 (python) S 5590 5592 5278 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 10 0 538487265 471572480 29537 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
[pid=5592/tid=5619] ppid=5590 vsize=460520 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5619/stat : 5619 (python) S 5590 5592 5278 0 -1 4202560 34 935 0 0 0 0 0 0 18 0 10 0 538487367 471572480 29537 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
[pid=5592/tid=5637] ppid=5590 vsize=460520 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5637/stat : 5637 (python) S 5590 5592 5278 0 -1 4202560 30 935 0 0 0 0 0 0 18 0 10 0 538516607 471572480 29537 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
[pid=5592/tid=5672] ppid=5590 vsize=460520 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5672/stat : 5672 (python) S 5590 5592 5278 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 10 0 538565825 471572480 29537 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
[pid=5592/tid=5737] ppid=5590 vsize=460520 CPUtime=0 cores=0,2,4,6
/proc/5592/task/5737/stat : 5737 (python) S 5590 5592 5278 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 10 0 538594512 471572480 29537 33554432000 4194304 5683644 140735875111200 18446744073709551615 248101521841 0 0 16781312 2 18446604450926846976 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.39
Current children cumulated vsize (KiB) 13565936

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 5592 and gives
#  childrusage.ru_utime.tv_sec=24
#  childrusage.ru_utime.tv_usec=456282
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=247962
# CPU time returned by wait4() is 24.7042
# while last known CPU time is 4800.39
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1221.67
CPU time (s): 4800.39
CPU user time (s): 4791.42
CPU system time (s): 8.97
CPU usage (%): 392.936
Max. virtual memory (cumulated for all children) (KiB): 13684356

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 24.4563
system time used= 0.247962
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 52247
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3004
involuntary context switches= 347

runsolver used 3.21451 second user time and 7.53385 second system time

The end

Launcher Data 

Begin job on node123 at 2011-05-01 18:52:31
IDJOB=3317928
IDBENCH=82390
IDSOLVER=1706
FILE ID=node123/3317928-1304268750
RUNJOBID= node123-1304267283-5296
PBS_JOBID= 13173956
Free space on /tmp= 73432 MiB

SOLVER NAME= borg-sat 11.03.02-industrial
BENCH NAME= SAT11/application/rintanen/SATPlanning/openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_1.025-NOTKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317928-1304268750/watcher-3317928-1304268750 -o /tmp/evaluation-result-3317928-1304268750/solver-3317928-1304268750 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317928-1304268750.cnf -seed 1512910626 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 732b9c9b0d44d6a9e75489b2f5004165
RANDOM SEED=1512910626

node123.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      29532248 kB
Buffers:        351188 kB
Cached:        1696072 kB
SwapCached:        100 kB
Active:        1180480 kB
Inactive:      1876800 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      29532248 kB
SwapTotal:    67111528 kB
SwapFree:     67111428 kB
Dirty:           39656 kB
Writeback:          80 kB
AnonPages:     1010364 kB
Mapped:          23560 kB
Slab:           289328 kB
PageTables:      10896 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  1566580 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73384 MiB
End job on node123 at 2011-05-01 19:12:55