Trace number 3310292

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 40000.4 5003.92

General information on the benchmark

Nameapplication/jarvisalo/HWMCC10-combination-timeframe-expansion/
hwmcc10-timeframe-expansion-k50-eijkbs6669-tseitin.cnf
MD5SUM7886cd26d6e0801ea382a1ffd6224023
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark2052.83
Satisfiable
(Un)Satisfiability was proved
Number of variables192503
Number of clauses564953
Sum of the clauses size1318221
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2376634
Number of clauses of size 3188317
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3310292-1304008190/watcher-3310292-1304008190 -o /tmp/evaluation-result-3310292-1304008190/solver-3310292-1304008190 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3310292-1304008190.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.52 7.89 7.71 2/170 15923
/proc/meminfo: memFree=11761924/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) R 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0

[startup+0.02738 s]
/proc/loadavg: 7.52 7.89 7.71 2/170 15923
/proc/meminfo: memFree=11761924/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 4 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100364 s]
/proc/loadavg: 7.52 7.89 7.71 2/170 15923
/proc/meminfo: memFree=11761924/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 4 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300329 s]
/proc/loadavg: 7.52 7.89 7.71 2/170 15923
/proc/meminfo: memFree=11761924/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 23 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700256 s]
/proc/loadavg: 7.52 7.89 7.71 2/170 15923
/proc/meminfo: memFree=11761924/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 19 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50109 s]
/proc/loadavg: 7.32 7.84 7.70 10/180 15938
/proc/meminfo: memFree=11260000/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 17 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
[pid=15924] ppid=15923 vsize=202604 CPUtime=1.48 cores=0
/proc/15924/stat : 15924 (cryptominisat) R 15923 15923 15667 0 -1 4202496 41310 0 0 0 144 4 0 0 25 0 1 0 591049737 207466496 36706 33554432000 4194304 4587354 140733733139904 18446744073709551615 216734875111 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15924/statm: 50651 36706 317 96 0 47213 0
[pid=15925] ppid=15923 vsize=125680 CPUtime=1.49 cores=1
/proc/15925/stat : 15925 (clasp) R 15923 15923 15667 0 -1 4202496 32827 0 0 0 145 4 0 0 20 0 1 0 591049737 128696320 27861 33554432000 4194304 4785740 140734226283072 18446744073709551615 4495610 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/15925/statm: 31420 27861 319 145 0 28465 0
[pid=15926] ppid=15923 vsize=491464 CPUtime=1.49 cores=2
/proc/15926/stat : 15926 (TNM) R 15923 15923 15667 0 -1 4202496 19689 0 0 0 147 2 0 0 20 0 1 0 591049737 503259136 19585 33554432000 134512640 134534564 4290266848 18446744073709551615 134529366 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/15926/statm: 122866 19585 95 6 0 121532 0
[pid=15927] ppid=15923 vsize=101176 CPUtime=1.49 cores=3
/proc/15927/stat : 15927 (march_hi) R 15923 15923 15667 0 -1 4202496 81947 0 0 0 139 10 0 0 20 0 1 0 591049737 103604224 18331 33554432000 4194304 4331988 140733862915344 18446744073709551615 216734839493 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/15927/statm: 25294 18335 140 34 0 23736 0
[pid=15928] ppid=15923 vsize=244912 CPUtime=3.87 cores=4-7
/proc/15928/stat : 15928 (plingeling) S 15923 15923 15667 0 -1 4202496 42993 1249 0 0 382 5 0 0 18 0 5 0 591049737 250789888 33316 33554432000 4194304 4356188 140736487957536 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/15928/statm: 61228 33316 147 40 0 59772 0
[pid=15928/tid=15935] ppid=15923 vsize=244912 CPUtime=0.79 cores=4-7
/proc/15928/task/15935/stat : 15935 (plingeling) R 15923 15923 15667 0 -1 4202560 2504 1249 0 0 79 0 0 0 18 0 5 0 591049807 250789888 33316 33554432000 4194304 4356188 140736487957536 18446744073709551615 4237251 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15928/tid=15936] ppid=15923 vsize=244912 CPUtime=0.79 cores=4-7
/proc/15928/task/15936/stat : 15936 (plingeling) R 15923 15923 15667 0 -1 4202560 2502 1249 0 0 79 0 0 0 18 0 5 0 591049807 250789888 33316 33554432000 4194304 4356188 140736487957536 18446744073709551615 4260258 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15928/tid=15937] ppid=15923 vsize=244912 CPUtime=0.79 cores=4-7
/proc/15928/task/15937/stat : 15937 (plingeling) R 15923 15923 15667 0 -1 4202560 2510 1249 0 0 79 0 0 0 18 0 5 0 591049807 250789888 33316 33554432000 4194304 4356188 140736487957536 18446744073709551615 4261931 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=15928/tid=15938] ppid=15923 vsize=244912 CPUtime=0.78 cores=4-7
/proc/15928/task/15938/stat : 15938 (plingeling) R 15923 15923 15667 0 -1 4202560 2533 1249 0 0 78 0 0 0 18 0 5 0 591049807 250789888 33316 33554432000 4194304 4356188 140736487957536 18446744073709551615 4237251 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 9.82
Current children cumulated vsize (KiB) 1184160

[startup+3.10081 s]
/proc/loadavg: 7.32 7.84 7.70 9/180 15938
/proc/meminfo: memFree=11197876/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
[pid=15924] ppid=15923 vsize=214456 CPUtime=3.09 cores=0
/proc/15924/stat : 15924 (cryptominisat) R 15923 15923 15667 0 -1 4202496 50166 0 0 0 303 6 0 0 25 0 1 0 591049737 219602944 39554 33554432000 4194304 4587354 140733733139904 18446744073709551615 4238719 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15924/statm: 53614 39554 331 96 0 50176 0
[pid=15925] ppid=15923 vsize=121972 CPUtime=3.09 cores=1
/proc/15925/stat : 15925 (clasp) R 15923 15923 15667 0 -1 4202496 35177 0 0 0 305 4 0 0 25 0 1 0 591049737 124899328 27227 33554432000 4194304 4785740 140734226283072 18446744073709551615 4319146 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/15925/statm: 30493 27227 340 145 0 27538 0
[pid=15926] ppid=15923 vsize=491464 CPUtime=3.09 cores=2
/proc/15926/stat : 15926 (TNM) R 15923 15923 15667 0 -1 4202496 19689 0 0 0 307 2 0 0 25 0 1 0 591049737 503259136 19585 33554432000 134512640 134534564 4290266848 18446744073709551615 134529352 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/15926/statm: 122866 19585 95 6 0 121532 0
[pid=15927] ppid=15923 vsize=367096 CPUtime=3.09 cores=3
/proc/15927/stat : 15927 (march_hi) R 15923 15923 15667 0 -1 4202496 161828 0 0 0 288 21 0 0 25 0 1 0 591049737 375906304 42389 33554432000 4194304 4331988 140733862915344 18446744073709551615 4229471 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/15927/statm: 91774 42389 157 34 0 90216 0
[pid=15928] ppid=15923 vsize=312600 CPUtime=10.25 cores=4-7
/proc/15928/stat : 15928 (plingeling) S 15923 15923 15667 0 -1 4202496 85416 1249 0 0 1015 10 0 0 18 0 5 0 591049737 320102400 33890 33554432000 4194304 4356188 140736487957536 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/15928/statm: 78150 33890 164 40 0 76694 0
[pid=15928/tid=15935] ppid=15923 vsize=312600 CPUtime=2.39 cores=4-7
/proc/15928/task/15935/stat : 15935 (plingeling) R 15923 15923 15667 0 -1 4202560 14451 1249 0 0 237 2 0 0 21 0 5 0 591049807 320102400 33890 33554432000 4194304 4356188 140736487957536 18446744073709551615 4275586 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15928/tid=15936] ppid=15923 vsize=312600 CPUtime=2.39 cores=4-7
/proc/15928/task/15936/stat : 15936 (plingeling) R 15923 15923 15667 0 -1 4202560 10741 1249 0 0 238 1 0 0 21 0 5 0 591049807 320102400 33890 33554432000 4194304 4356188 140736487957536 18446744073709551615 4303184 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15928/tid=15937] ppid=15923 vsize=312600 CPUtime=2.39 cores=4-7
/proc/15928/task/15937/stat : 15937 (plingeling) R 15923 15923 15667 0 -1 4202560 12602 1249 0 0 238 1 0 0 21 0 5 0 591049807 320102400 33890 33554432000 4194304 4356188 140736487957536 18446744073709551615 4218169 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=15928/tid=15938] ppid=15923 vsize=312600 CPUtime=2.37 cores=4-7
/proc/15928/task/15938/stat : 15938 (plingeling) R 15923 15923 15667 0 -1 4202560 14678 1249 0 0 236 1 0 0 22 0 5 0 591049807 320102400 33890 33554432000 4194304 4356188 140736487957536 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 22.61
Current children cumulated vsize (KiB) 1525912

[startup+6.30125 s]
/proc/loadavg: 7.32 7.84 7.70 9/180 15938
/proc/meminfo: memFree=10870020/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
[pid=15924] ppid=15923 vsize=216984 CPUtime=6.29 cores=0
/proc/15924/stat : 15924 (cryptominisat) R 15923 15923 15667 0 -1 4202496 52246 0 0 0 623 6 0 0 25 0 1 0 591049737 222191616 41607 33554432000 4194304 4587354 140733733139904 18446744073709551615 4243757 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15924/statm: 54246 41607 337 96 0 50808 0
[pid=15925] ppid=15923 vsize=121972 CPUtime=6.29 cores=1
/proc/15925/stat : 15925 (clasp) R 15923 15923 15667 0 -1 4202496 35177 0 0 0 625 4 0 0 25 0 1 0 591049737 124899328 27227 33554432000 4194304 4785740 140734226283072 18446744073709551615 4319104 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/15925/statm: 30493 27227 340 145 0 27538 0
[pid=15926] ppid=15923 vsize=491464 CPUtime=6.28 cores=2
/proc/15926/stat : 15926 (TNM) R 15923 15923 15667 0 -1 4202496 19689 0 0 0 626 2 0 0 25 0 1 0 591049737 503259136 19585 33554432000 134512640 134534564 4290266848 18446744073709551615 134529228 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/15926/statm: 122866 19585 95 6 0 121532 0
[pid=15927] ppid=15923 vsize=368648 CPUtime=6.29 cores=3
/proc/15927/stat : 15927 (march_hi) R 15923 15923 15667 0 -1 4202496 200232 0 0 0 603 26 0 0 25 0 1 0 591049737 377495552 80793 33554432000 4194304 4331988 140733862915344 18446744073709551615 4299854 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/15927/statm: 92162 80793 173 34 0 90604 0
[pid=15928] ppid=15923 vsize=396208 CPUtime=23.04 cores=4-7
/proc/15928/stat : 15928 (plingeling) S 15923 15923 15667 0 -1 4202496 106318 1249 0 0 2291 13 0 0 18 0 5 0 591049737 405716992 54792 33554432000 4194304 4356188 140736487957536 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/15928/statm: 99052 54792 164 40 0 97596 0
[pid=15928/tid=15935] ppid=15923 vsize=396208 CPUtime=5.58 cores=4-7
/proc/15928/task/15935/stat : 15935 (plingeling) R 15923 15923 15667 0 -1 4202560 18803 1249 0 0 556 2 0 0 25 0 5 0 591049807 405716992 54792 33554432000 4194304 4356188 140736487957536 18446744073709551615 4230192 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15928/tid=15936] ppid=15923 vsize=396208 CPUtime=5.59 cores=4-7
/proc/15928/task/15936/stat : 15936 (plingeling) R 15923 15923 15667 0 -1 4202560 17909 1249 0 0 557 2 0 0 20 0 5 0 591049807 405716992 54792 33554432000 4194304 4356188 140736487957536 18446744073709551615 4312069 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15928/tid=15937] ppid=15923 vsize=396208 CPUtime=5.59 cores=4-7
/proc/15928/task/15937/stat : 15937 (plingeling) R 15923 15923 15667 0 -1 4202560 19771 1249 0 0 557 2 0 0 25 0 5 0 591049807 405716992 54792 33554432000 4194304 4356188 140736487957536 18446744073709551615 4211827 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=15928/tid=15938] ppid=15923 vsize=396208 CPUtime=5.55 cores=4-7
/proc/15928/task/15938/stat : 15938 (plingeling) R 15923 15923 15667 0 -1 4202560 16891 1249 0 0 554 1 0 0 25 0 5 0 591049807 405716992 54792 33554432000 4194304 4356188 140736487957536 18446744073709551615 4262896 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 48.19
Current children cumulated vsize (KiB) 1613600

[startup+12.7011 s]
/proc/loadavg: 7.43 7.84 7.70 9/180 15938
/proc/meminfo: memFree=10863100/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
[pid=15924] ppid=15923 vsize=1812672 CPUtime=12.69 cores=0
/proc/15924/stat : 15924 (cryptominisat) R 15923 15923 15667 0 -1 4202496 57473 0 0 0 1261 8 0 0 25 0 1 0 591049737 1856176128 40506 33554432000 4194304 4587354 140733733139904 18446744073709551615 4521274 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15924/statm: 453168 40506 337 96 0 449730 0
[pid=15925] ppid=15923 vsize=121972 CPUtime=12.69 cores=1
/proc/15925/stat : 15925 (clasp) R 15923 15923 15667 0 -1 4202496 35177 0 0 0 1265 4 0 0 25 0 1 0 591049737 124899328 27227 33554432000 4194304 4785740 140734226283072 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/15925/statm: 30493 27227 340 145 0 27538 0
[pid=15926] ppid=15923 vsize=495300 CPUtime=12.68 cores=2
/proc/15926/stat : 15926 (TNM) R 15923 15923 15667 0 -1 4202496 19690 0 0 0 1266 2 0 0 25 0 1 0 591049737 507187200 19586 33554432000 134512640 134534564 4290266848 18446744073709551615 6188052 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/15926/statm: 123825 19586 95 6 0 122491 0
[pid=15927] ppid=15923 vsize=370776 CPUtime=12.69 cores=3
/proc/15927/stat : 15927 (march_hi) R 15923 15923 15667 0 -1 4202496 200800 0 0 0 1243 26 0 0 25 0 1 0 591049737 379674624 81361 33554432000 4194304 4331988 140733862915344 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/15927/statm: 92694 81361 173 34 0 91136 0
[pid=15928] ppid=15923 vsize=405060 CPUtime=48.6 cores=4-7
/proc/15928/stat : 15928 (plingeling) S 15923 15923 15667 0 -1 4202496 108531 1249 0 0 4847 13 0 0 18 0 5 0 591049737 414781440 57005 33554432000 4194304 4356188 140736487957536 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/15928/statm: 101265 57005 164 40 0 99809 0
[pid=15928/tid=15935] ppid=15923 vsize=405060 CPUtime=11.98 cores=4-7
/proc/15928/task/15935/stat : 15935 (plingeling) R 15923 15923 15667 0 -1 4202560 18803 1249 0 0 1196 2 0 0 25 0 5 0 591049807 414781440 57005 33554432000 4194304 4356188 140736487957536 18446744073709551615 4276747 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15928/tid=15936] ppid=15923 vsize=405060 CPUtime=11.99 cores=4-7
/proc/15928/task/15936/stat : 15936 (plingeling) R 15923 15923 15667 0 -1 4202560 17909 1249 0 0 1197 2 0 0 25 0 5 0 591049807 414781440 57005 33554432000 4194304 4356188 140736487957536 18446744073709551615 4215249 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15928/tid=15937] ppid=15923 vsize=405060 CPUtime=11.99 cores=4-7
/proc/15928/task/15937/stat : 15937 (plingeling) R 15923 15923 15667 0 -1 4202560 19771 1249 0 0 1197 2 0 0 25 0 5 0 591049807 414781440 57005 33554432000 4194304 4356188 140736487957536 18446744073709551615 4306802 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=15928/tid=15938] ppid=15923 vsize=405060 CPUtime=11.92 cores=4-7
/proc/15928/task/15938/stat : 15938 (plingeling) R 15923 15923 15667 0 -1 4202560 19104 1249 0 0 1190 2 0 0 25 0 5 0 591049807 414781440 57005 33554432000 4194304 4356188 140736487957536 18446744073709551615 4210605 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 99.35
Current children cumulated vsize (KiB) 3224104

[startup+25.5009 s]
/proc/loadavg: 7.51 7.85 7.70 9/180 15938
/proc/meminfo: memFree=10871252/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
[pid=15924] ppid=15923 vsize=1830784 CPUtime=25.49 cores=0
/proc/15924/stat : 15924 (cryptominisat) R 15923 15923 15667 0 -1 4202496 63908 0 0 0 2540 9 0 0 25 0 1 0 591049737 1874722816 46860 33554432000 4194304 4587354 140733733139904 18446744073709551615 4251402 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=15926] ppid=15923 vsize=495304 CPUtime=4841.91 cores=2
/proc/15926/stat : 15926 (TNM) R 15923 15923 15667 0 -1 4202496 19699 0 0 0 484188 3 0 0 25 0 1 0 591049737 507191296 19595 33554432000 134512640 134534564 4290266848 18446744073709551615 134530151 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/15926/statm: 123826 19595 103 6 0 122492 0
[pid=15927] ppid=15923 vsize=429280 CPUtime=4840.78 cores=3
/proc/15927/stat : 15927 (march_hi) R 15923 15923 15667 0 -1 4202496 212740 0 0 0 484045 33 0 0 25 0 1 0 591049737 439582720 93301 33554432000 4194304 4331988 140733862915344 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/15927/statm: 107320 93301 173 34 0 105762 0
[pid=15928] ppid=15923 vsize=819884 CPUtime=19341.1 cores=4-7
/proc/15928/stat : 15928 (plingeling) S 15923 15923 15667 0 -1 4202496 1637992 1249 0 0 1933898 207 0 0 18 0 5 0 591049737 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/15928/statm: 204971 160711 164 40 0 203515 0
[pid=15928/tid=15935] ppid=15923 vsize=819884 CPUtime=4841.7 cores=4-7
/proc/15928/task/15935/stat : 15935 (plingeling) R 15923 15923 15667 0 -1 4202560 443744 1249 0 0 484110 60 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331002 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15928/tid=15936] ppid=15923 vsize=819884 CPUtime=4841.68 cores=4-7
/proc/15928/task/15936/stat : 15936 (plingeling) R 15923 15923 15667 0 -1 4202560 432778 1249 0 0 484110 58 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15928/tid=15937] ppid=15923 vsize=819884 CPUtime=4841.69 cores=4-7
/proc/15928/task/15937/stat : 15937 (plingeling) R 15923 15923 15667 0 -1 4202560 528233 1249 0 0 484107 62 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=15928/tid=15938] ppid=15923 vsize=819884 CPUtime=4815.27 cores=4-7
/proc/15928/task/15938/stat : 15938 (plingeling) R 15923 15923 15667 0 -1 4202560 200293 1249 0 0 481505 22 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38708.4
Current children cumulated vsize (KiB) 4945480

[startup+4902.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/180 16083
/proc/meminfo: memFree=9096524/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
[pid=15924] ppid=15923 vsize=2408160 CPUtime=4902.38 cores=0
/proc/15924/stat : 15924 (cryptominisat) R 15923 15923 15667 0 -1 4202496 2202933 0 0 0 489590 648 0 0 25 0 1 0 591049737 2465955840 195475 33554432000 4194304 4587354 140733733139904 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15924/statm: 602040 195475 337 96 0 598602 0
[pid=15925] ppid=15923 vsize=827488 CPUtime=4902.26 cores=1
/proc/15925/stat : 15925 (clasp) R 15923 15923 15667 0 -1 4202496 205346 0 0 0 490147 79 0 0 25 0 1 0 591049737 847347712 197396 33554432000 4194304 4785740 140734226283072 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/15925/statm: 206872 197396 340 145 0 203917 0
[pid=15926] ppid=15923 vsize=495304 CPUtime=4901.9 cores=2
/proc/15926/stat : 15926 (TNM) R 15923 15923 15667 0 -1 4202496 19699 0 0 0 490187 3 0 0 25 0 1 0 591049737 507191296 19595 33554432000 134512640 134534564 4290266848 18446744073709551615 134528661 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/15926/statm: 123826 19595 103 6 0 122492 0
[pid=15927] ppid=15923 vsize=429280 CPUtime=4900.78 cores=3
/proc/15927/stat : 15927 (march_hi) R 15923 15923 15667 0 -1 4202496 212837 0 0 0 490045 33 0 0 25 0 1 0 591049737 439582720 93398 33554432000 4194304 4331988 140733862915344 18446744073709551615 4228983 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/15927/statm: 107320 93398 173 34 0 105762 0
[pid=15928] ppid=15923 vsize=819884 CPUtime=19580.7 cores=4-7
/proc/15928/stat : 15928 (plingeling) S 15923 15923 15667 0 -1 4202496 1654376 1249 0 0 1957865 209 0 0 18 0 5 0 591049737 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/15928/statm: 204971 160711 164 40 0 203515 0
[pid=15928/tid=15935] ppid=15923 vsize=819884 CPUtime=4901.7 cores=4-7
/proc/15928/task/15935/stat : 15935 (plingeling) R 15923 15923 15667 0 -1 4202560 443744 1249 0 0 490110 60 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331062 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15928/tid=15936] ppid=15923 vsize=819884 CPUtime=4901.68 cores=4-7
/proc/15928/task/15936/stat : 15936 (plingeling) R 15923 15923 15667 0 -1 4202560 432778 1249 0 0 490110 58 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4234178 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15928/tid=15937] ppid=15923 vsize=819884 CPUtime=4901.69 cores=4-7
/proc/15928/task/15937/stat : 15937 (plingeling) R 15923 15923 15667 0 -1 4202560 544617 1249 0 0 490106 63 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=15928/tid=15938] ppid=15923 vsize=819884 CPUtime=4874.95 cores=4-7
/proc/15928/task/15938/stat : 15938 (plingeling) R 15923 15923 15667 0 -1 4202560 200293 1249 0 0 487473 22 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331086 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39188.1
Current children cumulated vsize (KiB) 4998440

[startup+4962.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/180 16085
/proc/meminfo: memFree=9101204/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
[pid=15924] ppid=15923 vsize=2373508 CPUtime=4962.39 cores=0
/proc/15924/stat : 15924 (cryptominisat) R 15923 15923 15667 0 -1 4202496 2219469 0 0 0 495585 654 0 0 25 0 1 0 591049737 2430472192 194277 33554432000 4194304 4587354 140733733139904 18446744073709551615 4238512 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15924/statm: 593377 194277 337 96 0 589939 0
[pid=15925] ppid=15923 vsize=827488 CPUtime=4962.26 cores=1
/proc/15925/stat : 15925 (clasp) R 15923 15923 15667 0 -1 4202496 205346 0 0 0 496147 79 0 0 25 0 1 0 591049737 847347712 197396 33554432000 4194304 4785740 140734226283072 18446744073709551615 4553411 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/15925/statm: 206872 197396 340 145 0 203917 0
[pid=15926] ppid=15923 vsize=495304 CPUtime=4961.9 cores=2
/proc/15926/stat : 15926 (TNM) R 15923 15923 15667 0 -1 4202496 19699 0 0 0 496187 3 0 0 25 0 1 0 591049737 507191296 19595 33554432000 134512640 134534564 4290266848 18446744073709551615 134528412 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/15926/statm: 123826 19595 103 6 0 122492 0
[pid=15927] ppid=15923 vsize=429280 CPUtime=4960.78 cores=3
/proc/15927/stat : 15927 (march_hi) R 15923 15923 15667 0 -1 4202496 212930 0 0 0 496045 33 0 0 25 0 1 0 591049737 439582720 93491 33554432000 4194304 4331988 140733862915344 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/15927/statm: 107320 93491 173 34 0 105762 0
[pid=15928] ppid=15923 vsize=819884 CPUtime=19820.4 cores=4-7
/proc/15928/stat : 15928 (plingeling) S 15923 15923 15667 0 -1 4202496 1678986 1249 0 0 1981829 212 0 0 18 0 5 0 591049737 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/15928/statm: 204971 160711 164 40 0 203515 0
[pid=15928/tid=15935] ppid=15923 vsize=819884 CPUtime=4961.7 cores=4-7
/proc/15928/task/15935/stat : 15935 (plingeling) R 15923 15923 15667 0 -1 4202560 460128 1249 0 0 496108 62 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15928/tid=15936] ppid=15923 vsize=819884 CPUtime=4961.67 cores=4-7
/proc/15928/task/15936/stat : 15936 (plingeling) R 15923 15923 15667 0 -1 4202560 440970 1249 0 0 496108 59 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4332165 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15928/tid=15937] ppid=15923 vsize=819884 CPUtime=4961.69 cores=4-7
/proc/15928/task/15937/stat : 15937 (plingeling) R 15923 15923 15667 0 -1 4202560 544651 1249 0 0 496106 63 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=15928/tid=15938] ppid=15923 vsize=819884 CPUtime=4934.62 cores=4-7
/proc/15928/task/15938/stat : 15938 (plingeling) R 15923 15923 15667 0 -1 4202560 200293 1249 0 0 493440 22 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331392 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39667.7
Current children cumulated vsize (KiB) 4963788



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.91 s]
/proc/loadavg: 8.06 8.02 8.00 9/180 16086
/proc/meminfo: memFree=9102660/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
[pid=15924] ppid=15923 vsize=2372860 CPUtime=5003.99 cores=0
/proc/15924/stat : 15924 (cryptominisat) R 15923 15923 15667 0 -1 4202496 2236283 0 0 0 499741 658 0 0 25 0 1 0 591049737 2429808640 193738 33554432000 4194304 4587354 140733733139904 18446744073709551615 4238525 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15924/statm: 593215 193738 337 96 0 589777 0
[pid=15925] ppid=15923 vsize=827488 CPUtime=5003.87 cores=1
/proc/15925/stat : 15925 (clasp) R 15923 15923 15667 0 -1 4202496 205346 0 0 0 500308 79 0 0 25 0 1 0 591049737 847347712 197396 33554432000 4194304 4785740 140734226283072 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/15925/statm: 206872 197396 340 145 0 203917 0
[pid=15926] ppid=15923 vsize=495304 CPUtime=5003.51 cores=2
/proc/15926/stat : 15926 (TNM) R 15923 15923 15667 0 -1 4202496 19699 0 0 0 500348 3 0 0 25 0 1 0 591049737 507191296 19595 33554432000 134512640 134534564 4290266848 18446744073709551615 134530614 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/15926/statm: 123826 19595 103 6 0 122492 0
[pid=15927] ppid=15923 vsize=429280 CPUtime=5002.39 cores=3
/proc/15927/stat : 15927 (march_hi) R 15923 15923 15667 0 -1 4202496 212995 0 0 0 500206 33 0 0 25 0 1 0 591049737 439582720 93556 33554432000 4194304 4331988 140733862915344 18446744073709551615 4263258 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/15927/statm: 107320 93556 173 34 0 105762 0
[pid=15928] ppid=15923 vsize=819884 CPUtime=19986.6 cores=4-7
/proc/15928/stat : 15928 (plingeling) S 15923 15923 15667 0 -1 4202496 1695370 1249 0 0 1998449 214 0 0 18 0 5 0 591049737 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/15928/statm: 204971 160711 164 40 0 203515 0
[pid=15928/tid=15935] ppid=15923 vsize=819884 CPUtime=5003.31 cores=4-7
/proc/15928/task/15935/stat : 15935 (plingeling) R 15923 15923 15667 0 -1 4202560 460128 1249 0 0 500269 62 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15928/tid=15936] ppid=15923 vsize=819884 CPUtime=5003.29 cores=4-7
/proc/15928/task/15936/stat : 15936 (plingeling) R 15923 15923 15667 0 -1 4202560 440970 1249 0 0 500270 59 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15928/tid=15937] ppid=15923 vsize=819884 CPUtime=5003.3 cores=4-7
/proc/15928/task/15937/stat : 15937 (plingeling) R 15923 15923 15667 0 -1 4202560 561035 1249 0 0 500265 65 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=15928/tid=15938] ppid=15923 vsize=819884 CPUtime=4976 cores=4-7
/proc/15928/task/15938/stat : 15938 (plingeling) R 15923 15923 15667 0 -1 4202560 200293 1249 0 0 497578 22 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4241816 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 4963140

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5003.91 s]
/proc/loadavg: 8.06 8.02 8.00 9/180 16086
/proc/meminfo: memFree=9102660/32951124 swapFree=67102140/67111528
[pid=15923] ppid=15921 vsize=18324 CPUtime=0 cores=0-7
/proc/15923/stat : 15923 (ppfolio) S 15921 15923 15667 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 591049737 18763776 390 33554432000 4194304 4287625 140734347461968 18446744073709551615 216735198735 0 0 4096 65536 8068471003296038911 0 0 17 6 0 0 0
/proc/15923/statm: 4581 390 341 23 0 91 0
[pid=15924] ppid=15923 vsize=2372860 CPUtime=5003.99 cores=0
/proc/15924/stat : 15924 (cryptominisat) R 15923 15923 15667 0 -1 4202496 2236283 0 0 0 499741 658 0 0 25 0 1 0 591049737 2429808640 193738 33554432000 4194304 4587354 140733733139904 18446744073709551615 4238525 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15924/statm: 593215 193738 337 96 0 589777 0
[pid=15925] ppid=15923 vsize=827488 CPUtime=5003.87 cores=1
/proc/15925/stat : 15925 (clasp) R 15923 15923 15667 0 -1 4202496 205346 0 0 0 500308 79 0 0 25 0 1 0 591049737 847347712 197396 33554432000 4194304 4785740 140734226283072 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/15925/statm: 206872 197396 340 145 0 203917 0
[pid=15926] ppid=15923 vsize=495304 CPUtime=5003.51 cores=2
/proc/15926/stat : 15926 (TNM) R 15923 15923 15667 0 -1 4202496 19699 0 0 0 500348 3 0 0 25 0 1 0 591049737 507191296 19595 33554432000 134512640 134534564 4290266848 18446744073709551615 134530614 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/15926/statm: 123826 19595 103 6 0 122492 0
[pid=15927] ppid=15923 vsize=429280 CPUtime=5002.39 cores=3
/proc/15927/stat : 15927 (march_hi) R 15923 15923 15667 0 -1 4202496 212995 0 0 0 500206 33 0 0 25 0 1 0 591049737 439582720 93556 33554432000 4194304 4331988 140733862915344 18446744073709551615 4263258 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/15927/statm: 107320 93556 173 34 0 105762 0
[pid=15928] ppid=15923 vsize=819884 CPUtime=19986.6 cores=4-7
/proc/15928/stat : 15928 (plingeling) S 15923 15923 15667 0 -1 4202496 1695370 1249 0 0 1998449 214 0 0 18 0 5 0 591049737 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/15928/statm: 204971 160711 164 40 0 203515 0
[pid=15928/tid=15935] ppid=15923 vsize=819884 CPUtime=5003.31 cores=4-7
/proc/15928/task/15935/stat : 15935 (plingeling) R 15923 15923 15667 0 -1 4202560 460128 1249 0 0 500269 62 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=15928/tid=15936] ppid=15923 vsize=819884 CPUtime=5003.29 cores=4-7
/proc/15928/task/15936/stat : 15936 (plingeling) R 15923 15923 15667 0 -1 4202560 440970 1249 0 0 500270 59 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=15928/tid=15937] ppid=15923 vsize=819884 CPUtime=5003.3 cores=4-7
/proc/15928/task/15937/stat : 15937 (plingeling) R 15923 15923 15667 0 -1 4202560 561035 1249 0 0 500265 65 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=15928/tid=15938] ppid=15923 vsize=819884 CPUtime=4976 cores=4-7
/proc/15928/task/15938/stat : 15938 (plingeling) R 15923 15923 15667 0 -1 4202560 200293 1249 0 0 497578 22 0 0 25 0 5 0 591049807 839561216 160711 33554432000 4194304 4356188 140736487957536 18446744073709551615 4241816 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 4963140

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 15923 and gives
#  childrusage.ru_utime.tv_sec=5003
#  childrusage.ru_utime.tv_usec=482355
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=46992
# CPU time returned by wait4() is 5003.53
# while last known CPU time is 40000.4
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5003.92
CPU time (s): 40000.4
CPU user time (s): 39990.5
CPU system time (s): 9.87
CPU usage (%): 799.381
Max. virtual memory (cumulated for all children) (KiB): 6986540

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5003.48
system time used= 0.046992
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 20223
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 379
involuntary context switches= 15465

runsolver used 7.8828 second user time and 19.421 second system time

The end

Launcher Data 

Begin job on node119 at 2011-04-28 18:29:51
IDJOB=3310292
IDBENCH=82493
IDSOLVER=1638
FILE ID=node119/3310292-1304008190
RUNJOBID= node119-1304004602-15681
PBS_JOBID= 13168820
Free space on /tmp= 71808 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/jarvisalo/HWMCC10-combination-timeframe-expansion/hwmcc10-timeframe-expansion-k50-eijkbs6669-tseitin.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3310292-1304008190/watcher-3310292-1304008190 -o /tmp/evaluation-result-3310292-1304008190/solver-3310292-1304008190 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3310292-1304008190.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 7886cd26d6e0801ea382a1ffd6224023
RANDOM SEED=125360620

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      11762080 kB
Buffers:       2241468 kB
Cached:       18300040 kB
SwapCached:       5504 kB
Active:        9471424 kB
Inactive:     11131096 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      11762080 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:           20376 kB
Writeback:           0 kB
AnonPages:       58236 kB
Mapped:          14368 kB
Slab:           520296 kB
PageTables:       4120 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   185296 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71796 MiB
End job on node119 at 2011-04-28 19:53:17