Trace number 3221985

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39972.2 5000.02

General information on the benchmark

Namerandom/medium/
unif-k5-r21.3-v150-c3195-S788239782-034.UNKNOWN.cnf
MD5SUMea72de9bc2f16b1185170420c6413a72
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables150
Number of clauses3195
Sum of the clauses size15975
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 53195
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3221985-1304162996/watcher-3221985-1304162996 -o /tmp/evaluation-result-3221985-1304162996/solver-3221985-1304162996 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3221985-1304162996.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.00 8.03 8.00 1/170 17964
/proc/meminfo: memFree=29484048/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) S 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 4222494915495460863 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0

[startup+0.052355 s]
/proc/loadavg: 8.00 8.03 8.00 1/170 17964
/proc/meminfo: memFree=29484048/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) R 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100347 s]
/proc/loadavg: 8.00 8.03 8.00 1/170 17964
/proc/meminfo: memFree=29484048/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) R 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261616384 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300311 s]
/proc/loadavg: 8.00 8.03 8.00 1/170 17964
/proc/meminfo: memFree=29484048/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) R 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700241 s]
/proc/loadavg: 8.00 8.03 8.00 1/170 17964
/proc/meminfo: memFree=29484048/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) S 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 23 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 4222494915495460863 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.5011 s]
/proc/loadavg: 8.00 8.03 8.00 9/180 17979
/proc/meminfo: memFree=29467600/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) S 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 4222494915495460863 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
[pid=17965] ppid=17964 vsize=3398420 CPUtime=1.49 cores=0
/proc/17965/stat : 17965 (cryptominisat) R 17964 17964 17607 0 -1 4202496 4326 0 0 0 149 0 0 0 20 0 1 0 508714905 3479982080 2835 33554432000 4194304 4587354 140737037543840 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17965/statm: 849605 2835 337 96 0 846167 0
[pid=17966] ppid=17964 vsize=14332 CPUtime=1.49 cores=1
/proc/17966/stat : 17966 (clasp) R 17964 17964 17607 0 -1 4202496 1049 0 0 0 149 0 0 0 20 0 1 0 508714905 14675968 939 33554432000 4194304 4785740 140735941855552 18446744073709551615 4287008 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17966/statm: 3583 939 340 145 0 628 0
[pid=17967] ppid=17964 vsize=456088 CPUtime=1.49 cores=2
/proc/17967/stat : 17967 (TNM) R 17964 17964 17607 0 -1 4202496 389 0 0 0 149 0 0 0 20 0 1 0 508714905 467034112 285 33554432000 134512640 134534564 4292832544 18446744073709551615 134528275 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17967/statm: 114022 285 95 6 0 112688 0
[pid=17968] ppid=17964 vsize=7940 CPUtime=1.49 cores=3
/proc/17968/stat : 17968 (march_hi) R 17964 17964 17607 0 -1 4202496 584 0 0 0 149 0 0 0 20 0 1 0 508714905 8130560 470 33554432000 4194304 4331988 140734059357520 18446744073709551615 4226560 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17968/statm: 1985 470 175 34 0 427 0
[pid=17969] ppid=17964 vsize=122072 CPUtime=5.93 cores=4-7
/proc/17969/stat : 17969 (plingeling) S 17964 17964 17607 0 -1 4202496 3515 1249 0 0 593 0 0 0 19 0 5 0 508714905 125001728 2594 33554432000 4194304 4356188 140734629148224 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/17969/statm: 30518 2594 162 40 0 29062 0
[pid=17969/tid=17976] ppid=17964 vsize=122072 CPUtime=1.48 cores=4-7
/proc/17969/task/17976/stat : 17976 (plingeling) R 17964 17964 17607 0 -1 4202560 668 1249 0 0 148 0 0 0 20 0 5 0 508714906 125001728 2594 33554432000 4194304 4356188 140734629148224 18446744073709551615 4330973 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17969/tid=17977] ppid=17964 vsize=122072 CPUtime=1.47 cores=4-7
/proc/17969/task/17977/stat : 17977 (plingeling) R 17964 17964 17607 0 -1 4202560 680 1249 0 0 147 0 0 0 23 0 5 0 508714906 125001728 2594 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17969/tid=17978] ppid=17964 vsize=122072 CPUtime=1.48 cores=4-7
/proc/17969/task/17978/stat : 17978 (plingeling) R 17964 17964 17607 0 -1 4202560 708 1249 0 0 148 0 0 0 23 0 5 0 508714906 125001728 2594 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17969/tid=17979] ppid=17964 vsize=122072 CPUtime=1.48 cores=4-7
/proc/17969/task/17979/stat : 17979 (plingeling) R 17964 17964 17607 0 -1 4202560 846 1249 0 0 148 0 0 0 25 0 5 0 508714906 125001728 2594 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.89
Current children cumulated vsize (KiB) 4017176

[startup+3.10081 s]
/proc/loadavg: 8.00 8.03 8.00 9/180 17979
/proc/meminfo: memFree=29455200/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) S 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 4222494915495460863 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
[pid=17965] ppid=17964 vsize=742468 CPUtime=3.09 cores=0
/proc/17965/stat : 17965 (cryptominisat) R 17964 17964 17607 0 -1 4202496 6322 0 0 0 309 0 0 0 25 0 1 0 508714905 760287232 3777 33554432000 4194304 4587354 140737037543840 18446744073709551615 4388345 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17965/statm: 185617 3777 337 96 0 182179 0
[pid=17966] ppid=17964 vsize=15032 CPUtime=3.09 cores=1
/proc/17966/stat : 17966 (clasp) R 17964 17964 17607 0 -1 4202496 1219 0 0 0 309 0 0 0 25 0 1 0 508714905 15392768 1109 33554432000 4194304 4785740 140735941855552 18446744073709551615 4287104 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17966/statm: 3758 1109 340 145 0 803 0
[pid=17967] ppid=17964 vsize=456088 CPUtime=3.09 cores=2
/proc/17967/stat : 17967 (TNM) R 17964 17964 17607 0 -1 4202496 389 0 0 0 309 0 0 0 25 0 1 0 508714905 467034112 285 33554432000 134512640 134534564 4292832544 18446744073709551615 9332214 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17967/statm: 114022 285 95 6 0 112688 0
[pid=17968] ppid=17964 vsize=8508 CPUtime=3.09 cores=3
/proc/17968/stat : 17968 (march_hi) R 17964 17964 17607 0 -1 4202496 680 0 0 0 309 0 0 0 25 0 1 0 508714905 8712192 566 33554432000 4194304 4331988 140734059357520 18446744073709551615 4226625 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17968/statm: 2127 566 175 34 0 569 0
[pid=17969] ppid=17964 vsize=126064 CPUtime=12.33 cores=4-7
/proc/17969/stat : 17969 (plingeling) S 17964 17964 17607 0 -1 4202496 5185 1249 0 0 1232 1 0 0 19 0 5 0 508714905 129089536 3592 33554432000 4194304 4356188 140734629148224 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/17969/statm: 31516 3592 162 40 0 30060 0
[pid=17969/tid=17976] ppid=17964 vsize=126064 CPUtime=3.08 cores=4-7
/proc/17969/task/17976/stat : 17976 (plingeling) R 17964 17964 17607 0 -1 4202560 1182 1249 0 0 308 0 0 0 25 0 5 0 508714906 129089536 3592 33554432000 4194304 4356188 140734629148224 18446744073709551615 4279432 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17969/tid=17977] ppid=17964 vsize=126064 CPUtime=3.07 cores=4-7
/proc/17969/task/17977/stat : 17977 (plingeling) R 17964 17964 17607 0 -1 4202560 905 1249 0 0 307 0 0 0 25 0 5 0 508714906 129089536 3592 33554432000 4194304 4356188 140734629148224 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17969/tid=17978] ppid=17964 vsize=126064 CPUtime=3.08 cores=4-7
/proc/17969/task/17978/stat : 17978 (plingeling) R 17964 17964 17607 0 -1 4202560 1125 1249 0 0 308 0 0 0 25 0 5 0 508714906 129089536 3592 33554432000 4194304 4356188 140734629148224 18446744073709551615 4330976 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17969/tid=17979] ppid=17964 vsize=126064 CPUtime=3.08 cores=4-7
/proc/17969/task/17979/stat : 17979 (plingeling) R 17964 17964 17607 0 -1 4202560 1360 1249 0 0 308 0 0 0 25 0 5 0 508714906 129089536 3592 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.69
Current children cumulated vsize (KiB) 1366484

[startup+6.30124 s]
/proc/loadavg: 8.00 8.03 8.00 9/180 17979
/proc/meminfo: memFree=29448132/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) S 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 4222494915495460863 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
[pid=17965] ppid=17964 vsize=3401900 CPUtime=6.29 cores=0
/proc/17965/stat : 17965 (cryptominisat) R 17964 17964 17607 0 -1 4202496 8553 0 0 0 629 0 0 0 25 0 1 0 508714905 3483545600 4061 33554432000 4194304 4587354 140737037543840 18446744073709551615 4238591 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17965/statm: 850475 4061 337 96 0 847037 0
[pid=17966] ppid=17964 vsize=16168 CPUtime=6.29 cores=1
/proc/17966/stat : 17966 (clasp) R 17964 17964 17607 0 -1 4202496 1474 0 0 0 629 0 0 0 25 0 1 0 508714905 16556032 1364 33554432000 4194304 4785740 140735941855552 18446744073709551615 4287008 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17966/statm: 4042 1364 340 145 0 1087 0
[pid=17967] ppid=17964 vsize=456088 CPUtime=6.29 cores=2
/proc/17967/stat : 17967 (TNM) R 17964 17964 17607 0 -1 4202496 389 0 0 0 629 0 0 0 25 0 1 0 508714905 467034112 285 33554432000 134512640 134534564 4292832544 18446744073709551615 134530652 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17967/statm: 114022 285 95 6 0 112688 0
[pid=17968] ppid=17964 vsize=9020 CPUtime=6.29 cores=3
/proc/17968/stat : 17968 (march_hi) R 17964 17964 17607 0 -1 4202496 871 0 0 0 629 0 0 0 25 0 1 0 508714905 9236480 757 33554432000 4194304 4331988 140734059357520 18446744073709551615 4239578 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17968/statm: 2255 757 175 34 0 697 0
[pid=17969] ppid=17964 vsize=128956 CPUtime=25.12 cores=4-7
/proc/17969/stat : 17969 (plingeling) S 17964 17964 17607 0 -1 4202496 7012 1249 0 0 2511 1 0 0 19 0 5 0 508714905 132050944 4299 33554432000 4194304 4356188 140734629148224 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/17969/statm: 32239 4299 162 40 0 30783 0
[pid=17969/tid=17976] ppid=17964 vsize=128956 CPUtime=6.28 cores=4-7
/proc/17969/task/17976/stat : 17976 (plingeling) R 17964 17964 17607 0 -1 4202560 1598 1249 0 0 628 0 0 0 25 0 5 0 508714906 132050944 4299 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17969/tid=17977] ppid=17964 vsize=128956 CPUtime=6.25 cores=4-7
/proc/17969/task/17977/stat : 17977 (plingeling) R 17964 17964 17607 0 -1 4202560 1562 1249 0 0 625 0 0 0 25 0 5 0 508714906 132050944 4299 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17969/tid=17978] ppid=17964 vsize=128956 CPUtime=6.28 cores=4-7
/proc/17969/task/17978/stat : 17978 (plingeling) R 17964 17964 17607 0 -1 4202560 1462 1249 0 0 628 0 0 0 25 0 5 0 508714906 132050944 4299 33554432000 4194304 4356188 140734629148224 18446744073709551615 4233920 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17969/tid=17979] ppid=17964 vsize=128956 CPUtime=6.28 cores=4-7
/proc/17969/task/17979/stat : 17979 (plingeling) R 17964 17964 17607 0 -1 4202560 1777 1249 0 0 628 0 0 0 25 0 5 0 508714906 132050944 4299 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.28
Current children cumulated vsize (KiB) 4030456

[startup+12.7011 s]
/proc/loadavg: 8.00 8.03 8.00 9/180 17979
/proc/meminfo: memFree=29439676/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) S 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 4222494915495460863 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
[pid=17965] ppid=17964 vsize=747312 CPUtime=12.69 cores=0
/proc/17965/stat : 17965 (cryptominisat) R 17964 17964 17607 0 -1 4202496 13501 0 0 0 1268 1 0 0 25 0 1 0 508714905 765247488 4761 33554432000 4194304 4587354 140737037543840 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17965/statm: 186828 4761 337 96 0 183390 0
[pid=17966] ppid=17964 vsize=17344 CPUtime=12.69 cores=1
/proc/17966/stat : 17966 (clasp) R 17964 17964 17607 0 -1 4202496 1741 0 0 0 1269 0 0 0 25 0 1 0 508714905 17760256 1631 33554432000 4194304 4785740 140735941855552 18446744073709551615 4286927 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17966/statm: 4336 1631 340 145 0 1381 0
[pid=17967] ppid=17964 vsize=456088 CPUtime=12.69 cores=2
/proc/17967/stat : 17967 (TNM) R 17964 17964 17607 0 -1 4202496 389 0 0 0 1269 0 0 0 25 0 1 0 508714905 467034112 285 33554432000 134512640 134534564 4292832544 18446744073709551615 134529031 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17967/statm: 114022 285 95 6 0 112688 0
[pid=17968] ppid=17964 vsize=10936 CPUtime=12.69 cores=3
/proc/17968/stat : 17968 (march_hi) R 17964 17964 17607 0 -1 4202496 1210 0 0 0 1269 0 0 0 25 0 1 0 508714905 11198464 1096 33554432000 4194304 4331988 140734059357520 18446744073709551615 4240039 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17968/statm: 2734 1096 175 34 0 1176 0
[pid=17969] ppid=17964 vsize=132036 CPUtime=50.67 cores=4-7
/proc/17969/stat : 17969 (plingeling) S 17964 17964 17607 0 -1 4202496 10214 1249 0 0 5066 1 0 0 19 0 5 0 508714905 135204864 5069 33554432000 4194304 4356188 140734629148224 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/17969/statm: 33009 5069 162 40 0 31553 0
[pid=17969/tid=17976] ppid=17964 vsize=132036 CPUtime=12.68 cores=4-7
/proc/17969/task/17976/stat : 17976 (plingeling) R 17964 17964 17607 0 -1 4202560 2750 1249 0 0 1268 0 0 0 25 0 5 0 508714906 135204864 5069 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331264 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17969/tid=17977] ppid=17964 vsize=132036 CPUtime=12.61 cores=4-7
/proc/17969/task/17977/stat : 17977 (plingeling) R 17964 17964 17607 0 -1 4202560 2202 1249 0 0 1261 0 0 0 25 0 5 0 508714906 135204864 5069 33554432000 4194304 4356188 140734629148224 18446744073709551615 4332007 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17969/tid=17978] ppid=17964 vsize=132036 CPUtime=12.67 cores=4-7
/proc/17969/task/17978/stat : 17978 (plingeling) R 17964 17964 17607 0 -1 4202560 2232 1249 0 0 1267 0 0 0 25 0 5 0 508714906 135204864 5069 33554432000 4194304 4356188 140734629148224 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17969/tid=17979] ppid=17964 vsize=132036 CPUtime=12.68 cores=4-7
/proc/17969/task/17979/stat : 17979 (plingeling) R 17964 17964 17607 0 -1 4202560 2417 1249 0 0 1268 0 0 0 25 0 5 0 508714906 135204864 5069 33554432000 4194304 4356188 140734629148224 18446744073709551615 4332023 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.43
Current children cumulated vsize (KiB) 1382040

[startup+25.5008 s]
/proc/loadavg: 8.00 8.03 8.00 9/180 17980
/proc/meminfo: memFree=29422416/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) S 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 4222494915495460863 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
[pid=17965] ppid=17964 vsize=758932 CPUtime=25.49 cores=0
/proc/17965/stat : 17965 (cryptominisat) R 17964 17964 17607 0 -1 4202496 21588 0 0 0 2547 2 0 0 25 0 1 0 508714905 777146368 6126 33554432000 4194304 4587354 140737037543840 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=17967] ppid=17964 vsize=456092 CPUtime=4782.41 cores=2
/proc/17967/stat : 17967 (TNM) R 17964 17964 17607 0 -1 4202496 398 0 0 0 478240 1 0 0 25 0 1 0 508714905 467038208 294 33554432000 134512640 134534564 4292832544 18446744073709551615 134528628 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17967/statm: 114023 294 103 6 0 112689 0
[pid=17968] ppid=17964 vsize=1077088 CPUtime=4782.4 cores=3
/proc/17968/stat : 17968 (march_hi) R 17964 17964 17607 0 -1 4202496 200783 0 0 0 478090 150 0 0 25 0 1 0 508714905 1102938112 200654 33554432000 4194304 4331988 140734059357520 18446744073709551615 4210516 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17968/statm: 269272 200654 175 34 0 267714 0
[pid=17969] ppid=17964 vsize=242008 CPUtime=19102.2 cores=4-7
/proc/17969/stat : 17969 (plingeling) S 17964 17964 17607 0 -1 4202496 700502 1249 0 0 1910104 111 0 0 19 0 5 0 508714905 247816192 32578 33554432000 4194304 4356188 140734629148224 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/17969/statm: 60502 32578 162 40 0 59046 0
[pid=17969/tid=17976] ppid=17964 vsize=242008 CPUtime=4781.82 cores=4-7
/proc/17969/task/17976/stat : 17976 (plingeling) R 17964 17964 17607 0 -1 4202560 170355 1249 0 0 478155 27 0 0 25 0 5 0 508714906 247816192 32578 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331309 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17969/tid=17977] ppid=17964 vsize=242008 CPUtime=4755.52 cores=4-7
/proc/17969/task/17977/stat : 17977 (plingeling) R 17964 17964 17607 0 -1 4202560 179806 1249 0 0 475525 27 0 0 25 0 5 0 508714906 247816192 32578 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17969/tid=17978] ppid=17964 vsize=242008 CPUtime=4782.38 cores=4-7
/proc/17969/task/17978/stat : 17978 (plingeling) R 17964 17964 17607 0 -1 4202560 170411 1249 0 0 478211 27 0 0 25 0 5 0 508714906 247816192 32578 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17969/tid=17979] ppid=17964 vsize=242008 CPUtime=4782.4 cores=4-7
/proc/17969/task/17979/stat : 17979 (plingeling) R 17964 17964 17607 0 -1 4202560 179317 1249 0 0 478211 29 0 0 25 0 5 0 508714906 247816192 32578 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38231.8
Current children cumulated vsize (KiB) 2747412

[startup+4842.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/180 18120
/proc/meminfo: memFree=28347816/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) S 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 4222494915495460863 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
[pid=17965] ppid=17964 vsize=1009744 CPUtime=4842.4 cores=0
/proc/17965/stat : 17965 (cryptominisat) R 17964 17964 17607 0 -1 4202496 400207 0 0 0 484071 169 0 0 25 0 1 0 508714905 1033977856 37311 33554432000 4194304 4587354 140737037543840 18446744073709551615 4237681 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17965/statm: 252436 37311 337 96 0 248998 0
[pid=17966] ppid=17964 vsize=45888 CPUtime=4842.41 cores=1
/proc/17966/stat : 17966 (clasp) R 17964 17964 17607 0 -1 4202496 8192 0 0 0 484239 2 0 0 25 0 1 0 508714905 46989312 8082 33554432000 4194304 4785740 140735941855552 18446744073709551615 4287244 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17966/statm: 11472 8082 341 145 0 8517 0
[pid=17967] ppid=17964 vsize=456092 CPUtime=4842.41 cores=2
/proc/17967/stat : 17967 (TNM) R 17964 17964 17607 0 -1 4202496 398 0 0 0 484240 1 0 0 25 0 1 0 508714905 467038208 294 33554432000 134512640 134534564 4292832544 18446744073709551615 134530522 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17967/statm: 114023 294 103 6 0 112689 0
[pid=17968] ppid=17964 vsize=1077088 CPUtime=4842.41 cores=3
/proc/17968/stat : 17968 (march_hi) R 17964 17964 17607 0 -1 4202496 202917 0 0 0 484089 152 0 0 25 0 1 0 508714905 1102938112 202788 33554432000 4194304 4331988 140734059357520 18446744073709551615 4210416 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17968/statm: 269272 202788 175 34 0 267714 0
[pid=17969] ppid=17964 vsize=259288 CPUtime=19341.8 cores=4-7
/proc/17969/stat : 17969 (plingeling) S 17964 17964 17607 0 -1 4202496 708918 1249 0 0 1934070 112 0 0 19 0 5 0 508714905 265510912 36898 33554432000 4194304 4356188 140734629148224 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/17969/statm: 64822 36898 162 40 0 63366 0
[pid=17969/tid=17976] ppid=17964 vsize=259288 CPUtime=4841.81 cores=4-7
/proc/17969/task/17976/stat : 17976 (plingeling) R 17964 17964 17607 0 -1 4202560 171251 1249 0 0 484154 27 0 0 25 0 5 0 508714906 265510912 36898 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331281 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17969/tid=17977] ppid=17964 vsize=259288 CPUtime=4815.19 cores=4-7
/proc/17969/task/17977/stat : 17977 (plingeling) R 17964 17964 17607 0 -1 4202560 181022 1249 0 0 481492 27 0 0 25 0 5 0 508714906 265510912 36898 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331792 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17969/tid=17978] ppid=17964 vsize=259288 CPUtime=4842.38 cores=4-7
/proc/17969/task/17978/stat : 17978 (plingeling) R 17964 17964 17607 0 -1 4202560 171403 1249 0 0 484211 27 0 0 25 0 5 0 508714906 265510912 36898 33554432000 4194304 4356188 140734629148224 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17969/tid=17979] ppid=17964 vsize=259288 CPUtime=4842.4 cores=4-7
/proc/17969/task/17979/stat : 17979 (plingeling) R 17964 17964 17607 0 -1 4202560 184629 1249 0 0 484211 29 0 0 25 0 5 0 508714906 265510912 36898 33554432000 4194304 4356188 140734629148224 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38711.4
Current children cumulated vsize (KiB) 2866424

[startup+4902.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/180 18122
/proc/meminfo: memFree=28331412/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) S 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 4222494915495460863 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
[pid=17965] ppid=17964 vsize=906032 CPUtime=4902.4 cores=0
/proc/17965/stat : 17965 (cryptominisat) R 17964 17964 17607 0 -1 4202496 405946 0 0 0 490068 172 0 0 25 0 1 0 508714905 927776768 35264 33554432000 4194304 4587354 140737037543840 18446744073709551615 4238463 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17965/statm: 226508 35264 337 96 0 223070 0
[pid=17966] ppid=17964 vsize=45888 CPUtime=4902.41 cores=1
/proc/17966/stat : 17966 (clasp) R 17964 17964 17607 0 -1 4202496 8215 0 0 0 490239 2 0 0 25 0 1 0 508714905 46989312 8105 33554432000 4194304 4785740 140735941855552 18446744073709551615 4286964 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17966/statm: 11472 8105 341 145 0 8517 0
[pid=17967] ppid=17964 vsize=456092 CPUtime=4902.41 cores=2
/proc/17967/stat : 17967 (TNM) R 17964 17964 17607 0 -1 4202496 398 0 0 0 490240 1 0 0 25 0 1 0 508714905 467038208 294 33554432000 134512640 134534564 4292832544 18446744073709551615 134528687 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17967/statm: 114023 294 103 6 0 112689 0
[pid=17968] ppid=17964 vsize=1077088 CPUtime=4902.41 cores=3
/proc/17968/stat : 17968 (march_hi) R 17964 17964 17607 0 -1 4202496 205537 0 0 0 490087 154 0 0 25 0 1 0 508714905 1102938112 205408 33554432000 4194304 4331988 140734059357520 18446744073709551615 4287604 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17968/statm: 269272 205408 175 34 0 267714 0
[pid=17969] ppid=17964 vsize=271192 CPUtime=19581.5 cores=4-7
/proc/17969/stat : 17969 (plingeling) S 17964 17964 17607 0 -1 4202496 719350 1249 0 0 1958036 114 0 0 19 0 5 0 508714905 277700608 39874 33554432000 4194304 4356188 140734629148224 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/17969/statm: 67798 39874 162 40 0 66342 0
[pid=17969/tid=17976] ppid=17964 vsize=271192 CPUtime=4901.8 cores=4-7
/proc/17969/task/17976/stat : 17976 (plingeling) R 17964 17964 17607 0 -1 4202560 176307 1249 0 0 490152 28 0 0 25 0 5 0 508714906 277700608 39874 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17969/tid=17977] ppid=17964 vsize=271192 CPUtime=4874.86 cores=4-7
/proc/17969/task/17977/stat : 17977 (plingeling) R 17964 17964 17607 0 -1 4202560 185758 1249 0 0 487459 27 0 0 25 0 5 0 508714906 277700608 39874 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331126 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17969/tid=17978] ppid=17964 vsize=271192 CPUtime=4902.39 cores=4-7
/proc/17969/task/17978/stat : 17978 (plingeling) R 17964 17964 17607 0 -1 4202560 171403 1249 0 0 490212 27 0 0 25 0 5 0 508714906 277700608 39874 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17969/tid=17979] ppid=17964 vsize=271192 CPUtime=4902.4 cores=4-7
/proc/17969/task/17979/stat : 17979 (plingeling) R 17964 17964 17607 0 -1 4202560 185269 1249 0 0 490211 29 0 0 25 0 5 0 508714906 277700608 39874 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331203 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39191.1
Current children cumulated vsize (KiB) 2774616

[startup+4962.3 s]
/proc/loadavg: 8.00 8.00 8.00 9/180 18123
/proc/meminfo: memFree=28325424/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) S 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 4222494915495460863 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
[pid=17965] ppid=17964 vsize=906032 CPUtime=4962.4 cores=0
/proc/17965/stat : 17965 (cryptominisat) R 17964 17964 17607 0 -1 4202496 409770 0 0 0 496065 175 0 0 25 0 1 0 508714905 927776768 38977 33554432000 4194304 4587354 140737037543840 18446744073709551615 4238525 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17965/statm: 226508 38977 337 96 0 223070 0
[pid=17966] ppid=17964 vsize=45888 CPUtime=4962.41 cores=1
/proc/17966/stat : 17966 (clasp) R 17964 17964 17607 0 -1 4202496 8217 0 0 0 496239 2 0 0 25 0 1 0 508714905 46989312 8107 33554432000 4194304 4785740 140735941855552 18446744073709551615 4287011 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17966/statm: 11472 8107 341 145 0 8517 0
[pid=17967] ppid=17964 vsize=456092 CPUtime=4962.41 cores=2
/proc/17967/stat : 17967 (TNM) R 17964 17964 17607 0 -1 4202496 398 0 0 0 496240 1 0 0 25 0 1 0 508714905 467038208 294 33554432000 134512640 134534564 4292832544 18446744073709551615 134528708 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17967/statm: 114023 294 103 6 0 112689 0
[pid=17968] ppid=17964 vsize=1077088 CPUtime=4962.41 cores=3
/proc/17968/stat : 17968 (march_hi) R 17964 17964 17607 0 -1 4202496 208034 0 0 0 496085 156 0 0 25 0 1 0 508714905 1102938112 207905 33554432000 4194304 4331988 140734059357520 18446744073709551615 4287514 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17968/statm: 269272 207905 175 34 0 267714 0
[pid=17969] ppid=17964 vsize=254552 CPUtime=19821.2 cores=4-7
/proc/17969/stat : 17969 (plingeling) S 17964 17964 17607 0 -1 4202496 723382 1249 0 0 1982002 114 0 0 19 0 5 0 508714905 260661248 35714 33554432000 4194304 4356188 140734629148224 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/17969/statm: 63638 35714 162 40 0 62182 0
[pid=17969/tid=17976] ppid=17964 vsize=254552 CPUtime=4961.8 cores=4-7
/proc/17969/task/17976/stat : 17976 (plingeling) R 17964 17964 17607 0 -1 4202560 176691 1249 0 0 496151 29 0 0 25 0 5 0 508714906 260661248 35714 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17969/tid=17977] ppid=17964 vsize=254552 CPUtime=4934.53 cores=4-7
/proc/17969/task/17977/stat : 17977 (plingeling) R 17964 17964 17607 0 -1 4202560 186654 1249 0 0 493426 27 0 0 25 0 5 0 508714906 260661248 35714 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17969/tid=17978] ppid=17964 vsize=254552 CPUtime=4962.38 cores=4-7
/proc/17969/task/17978/stat : 17978 (plingeling) R 17964 17964 17607 0 -1 4202560 173259 1249 0 0 496211 27 0 0 25 0 5 0 508714906 260661248 35714 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331028 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17969/tid=17979] ppid=17964 vsize=254552 CPUtime=4962.4 cores=4-7
/proc/17969/task/17979/stat : 17979 (plingeling) R 17964 17964 17607 0 -1 4202560 186165 1249 0 0 496211 29 0 0 25 0 5 0 508714906 260661248 35714 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39670.8
Current children cumulated vsize (KiB) 2757976



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.06 8.02 8.00 9/180 18124
/proc/meminfo: memFree=28311536/32951124 swapFree=67111348/67111528
[pid=17964] ppid=17962 vsize=18324 CPUtime=0 cores=0-7
/proc/17964/stat : 17964 (ppfolio) S 17962 17964 17607 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 508714904 18763776 390 33554432000 4194304 4287625 140735484421664 18446744073709551615 254261636623 0 0 4096 65536 4222494915495460863 0 0 17 6 0 0 0
/proc/17964/statm: 4581 390 341 23 0 91 0
[pid=17965] ppid=17964 vsize=955160 CPUtime=5000.11 cores=0
/proc/17965/stat : 17965 (cryptominisat) R 17964 17964 17607 0 -1 4202496 414388 0 0 0 499834 177 0 0 25 0 1 0 508714905 978083840 35883 33554432000 4194304 4587354 140737037543840 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17965/statm: 238790 35883 337 96 0 235352 0
[pid=17966] ppid=17964 vsize=46140 CPUtime=5000.12 cores=1
/proc/17966/stat : 17966 (clasp) R 17964 17964 17607 0 -1 4202496 8245 0 0 0 500010 2 0 0 25 0 1 0 508714905 47247360 8135 33554432000 4194304 4785740 140735941855552 18446744073709551615 4553366 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17966/statm: 11535 8135 341 145 0 8580 0
[pid=17967] ppid=17964 vsize=456092 CPUtime=5000.12 cores=2
/proc/17967/stat : 17967 (TNM) R 17964 17964 17607 0 -1 4202496 398 0 0 0 500011 1 0 0 25 0 1 0 508714905 467038208 294 33554432000 134512640 134534564 4292832544 18446744073709551615 134528223 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17967/statm: 114023 294 103 6 0 112689 0
[pid=17968] ppid=17964 vsize=1077088 CPUtime=5000.11 cores=3
/proc/17968/stat : 17968 (march_hi) R 17964 17964 17607 0 -1 4202496 209403 0 0 0 499854 157 0 0 25 0 1 0 508714905 1102938112 209274 33554432000 4194304 4331988 140734059357520 18446744073709551615 4308792 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17968/statm: 269272 209274 175 34 0 267714 0
[pid=17969] ppid=17964 vsize=274520 CPUtime=19971.8 cores=4-7
/proc/17969/stat : 17969 (plingeling) S 17964 17964 17607 0 -1 4202496 730230 1249 0 0 1997062 115 0 0 19 0 5 0 508714905 281108480 40706 33554432000 4194304 4356188 140734629148224 18446744073709551615 254273419237 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/17969/statm: 68630 40706 162 40 0 67174 0
[pid=17969/tid=17976] ppid=17964 vsize=274520 CPUtime=4999.51 cores=4-7
/proc/17969/task/17976/stat : 17976 (plingeling) R 17964 17964 17607 0 -1 4202560 177523 1249 0 0 499922 29 0 0 25 0 5 0 508714906 281108480 40706 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17969/tid=17977] ppid=17964 vsize=274520 CPUtime=4972.04 cores=4-7
/proc/17969/task/17977/stat : 17977 (plingeling) R 17964 17964 17607 0 -1 4202560 187614 1249 0 0 497176 28 0 0 25 0 5 0 508714906 281108480 40706 33554432000 4194304 4356188 140734629148224 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17969/tid=17978] ppid=17964 vsize=274520 CPUtime=5000.09 cores=4-7
/proc/17969/task/17978/stat : 17978 (plingeling) R 17964 17964 17607 0 -1 4202560 173259 1249 0 0 499982 27 0 0 25 0 5 0 508714906 281108480 40706 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331340 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17969/tid=17979] ppid=17964 vsize=274520 CPUtime=5000.11 cores=4-7
/proc/17969/task/17979/stat : 17979 (plingeling) R 17964 17964 17607 0 -1 4202560 191221 1249 0 0 499980 31 0 0 25 0 5 0 508714906 281108480 40706 33554432000 4194304 4356188 140734629148224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39972.2
Current children cumulated vsize (KiB) 2827324

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 17964 and gives
#  childrusage.ru_utime.tv_sec=14998
#  childrusage.ru_utime.tv_usec=567871
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=826722
# CPU time returned by wait4() is 15000.4
# while last known CPU time is 39972.2
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39972.2
CPU user time (s): 39967.7
CPU system time (s): 4.52
CPU usage (%): 799.442
Max. virtual memory (cumulated for all children) (KiB): 7906184

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14998.6
system time used= 1.82672
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 423568
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 806
involuntary context switches= 30464

runsolver used 8.49671 second user time and 18.7831 second system time

The end

Launcher Data 

Begin job on node140 at 2011-04-30 13:29:56
IDJOB=3221985
IDBENCH=83014
IDSOLVER=1638
FILE ID=node140/3221985-1304162996
RUNJOBID= node140-1304158690-17621
PBS_JOBID= 13170918
Free space on /tmp= 73296 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/medium/unif-k5-r21.3-v150-c3195-S788239782-034.UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3221985-1304162996/watcher-3221985-1304162996 -o /tmp/evaluation-result-3221985-1304162996/solver-3221985-1304162996 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3221985-1304162996.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= ea72de9bc2f16b1185170420c6413a72
RANDOM SEED=1856374237

node140.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29483956 kB
Buffers:       1058188 kB
Cached:        1847648 kB
SwapCached:        180 kB
Active:         342764 kB
Inactive:      2625176 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29483956 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:           10180 kB
Writeback:           0 kB
AnonPages:       62224 kB
Mapped:          14760 kB
Slab:           432424 kB
PageTables:       4516 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   183524 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73284 MiB
End job on node140 at 2011-04-30 14:53:18