Trace number 3220071

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39972.3 5000.02

General information on the benchmark

Namerandom/large/
unif-k7-r85-v250-c21250-S977260632-016-UNKNOWN.cnf
MD5SUM58eb79e3bfc8d25580f794d4e2d18629
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables250
Number of clauses21250
Sum of the clauses size148750
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 521250

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3220071-1304155694/watcher-3220071-1304155694 -o /tmp/evaluation-result-3220071-1304155694/solver-3220071-1304155694 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3220071-1304155694.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.19 8.16 8.10 2/174 17311
/proc/meminfo: memFree=29981532/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 2 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0

[startup+0.0343259 s]
/proc/loadavg: 8.19 8.16 8.10 2/174 17311
/proc/meminfo: memFree=29981532/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.152269 s]
/proc/loadavg: 8.19 8.16 8.10 2/174 17311
/proc/meminfo: memFree=29981532/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300244 s]
/proc/loadavg: 8.19 8.16 8.10 2/174 17311
/proc/meminfo: memFree=29981532/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.70017 s]
/proc/loadavg: 8.19 8.16 8.10 2/174 17311
/proc/meminfo: memFree=29981532/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 20 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50102 s]
/proc/loadavg: 8.18 8.15 8.10 9/184 17326
/proc/meminfo: memFree=29948344/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
[pid=17312] ppid=17311 vsize=782556 CPUtime=1.49 cores=0
/proc/17312/stat : 17312 (cryptominisat) R 17311 17311 16810 0 -1 4202496 4937 0 0 0 148 1 0 0 20 0 1 0 527167574 801337344 1897 33554432000 4194304 4587354 140733322103776 18446744073709551615 4238571 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17312/statm: 195639 1897 327 96 0 192201 0
[pid=17313] ppid=17311 vsize=19676 CPUtime=1.48 cores=1
/proc/17313/stat : 17313 (clasp) R 17311 17311 16810 0 -1 4202496 2448 0 0 0 148 0 0 0 20 0 1 0 527167574 20148224 2259 33554432000 4194304 4785740 140733894257840 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17313/statm: 4919 2259 340 145 0 1964 0
[pid=17314] ppid=17311 vsize=454504 CPUtime=1.49 cores=2
/proc/17314/stat : 17314 (TNM) R 17311 17311 16810 0 -1 4202496 998 0 0 0 149 0 0 0 20 0 1 0 527167574 465412096 894 33554432000 134512640 134534564 4292253504 18446744073709551615 134524541 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17314/statm: 113626 894 91 6 0 112292 0
[pid=17315] ppid=17311 vsize=10204 CPUtime=1.49 cores=3
/proc/17315/stat : 17315 (march_hi) R 17311 17311 16810 0 -1 4202496 3931 0 0 0 149 0 0 0 20 0 1 0 527167574 10448896 918 33554432000 4194304 4331988 140736488535616 18446744073709551615 4226647 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17315/statm: 2551 918 150 34 0 993 0
[pid=17316] ppid=17311 vsize=124856 CPUtime=5.84 cores=4-7
/proc/17316/stat : 17316 (plingeling) S 17311 17311 16810 0 -1 4202496 6013 1250 0 0 583 1 0 0 18 0 5 0 527167574 127852544 3287 33554432000 4194304 4356188 140737347716640 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17316/statm: 31214 3287 162 40 0 29758 0
[pid=17316/tid=17323] ppid=17311 vsize=124856 CPUtime=1.45 cores=4-7
/proc/17316/task/17323/stat : 17323 (plingeling) R 17311 17311 16810 0 -1 4202560 842 1250 0 0 145 0 0 0 20 0 5 0 527167578 127852544 3287 33554432000 4194304 4356188 140737347716640 18446744073709551615 4330995 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17316/tid=17324] ppid=17311 vsize=124856 CPUtime=1.45 cores=4-7
/proc/17316/task/17324/stat : 17324 (plingeling) R 17311 17311 16810 0 -1 4202560 835 1250 0 0 145 0 0 0 20 0 5 0 527167578 127852544 3287 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17316/tid=17325] ppid=17311 vsize=124856 CPUtime=1.44 cores=4-7
/proc/17316/task/17325/stat : 17325 (plingeling) R 17311 17311 16810 0 -1 4202560 872 1250 0 0 144 0 0 0 23 0 5 0 527167578 127852544 3287 33554432000 4194304 4356188 140737347716640 18446744073709551615 4332165 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17316/tid=17326] ppid=17311 vsize=124856 CPUtime=1.44 cores=4-7
/proc/17316/task/17326/stat : 17326 (plingeling) R 17311 17311 16810 0 -1 4202560 923 1250 0 0 144 0 0 0 23 0 5 0 527167578 127852544 3287 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.79
Current children cumulated vsize (KiB) 1410120

[startup+3.10073 s]
/proc/loadavg: 8.18 8.15 8.10 9/184 17326
/proc/meminfo: memFree=29943384/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
[pid=17312] ppid=17311 vsize=800588 CPUtime=3.09 cores=0
/proc/17312/stat : 17312 (cryptominisat) R 17311 17311 16810 0 -1 4202496 10552 0 0 0 307 2 0 0 25 0 1 0 527167574 819802112 5683 33554432000 4194304 4587354 140733322103776 18446744073709551615 4388352 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17312/statm: 200147 5683 337 96 0 196709 0
[pid=17313] ppid=17311 vsize=21248 CPUtime=3.08 cores=1
/proc/17313/stat : 17313 (clasp) R 17311 17311 16810 0 -1 4202496 2834 0 0 0 308 0 0 0 25 0 1 0 527167574 21757952 2645 33554432000 4194304 4785740 140733894257840 18446744073709551615 4309576 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17313/statm: 5312 2645 340 145 0 2357 0
[pid=17314] ppid=17311 vsize=454504 CPUtime=3.08 cores=2
/proc/17314/stat : 17314 (TNM) R 17311 17311 16810 0 -1 4202496 998 0 0 0 308 0 0 0 25 0 1 0 527167574 465412096 894 33554432000 134512640 134534564 4292253504 18446744073709551615 134524573 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17314/statm: 113626 894 91 6 0 112292 0
[pid=17315] ppid=17311 vsize=13112 CPUtime=3.08 cores=3
/proc/17315/stat : 17315 (march_hi) R 17311 17311 16810 0 -1 4202496 4802 0 0 0 308 0 0 0 25 0 1 0 527167574 13426688 1476 33554432000 4194304 4331988 140736488535616 18446744073709551615 4287132 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17315/statm: 3278 1476 174 34 0 1720 0
[pid=17316] ppid=17311 vsize=127552 CPUtime=12.23 cores=4-7
/proc/17316/stat : 17316 (plingeling) S 17311 17311 16810 0 -1 4202496 6676 1250 0 0 1222 1 0 0 18 0 5 0 527167574 130613248 3950 33554432000 4194304 4356188 140737347716640 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17316/statm: 31888 3950 162 40 0 30432 0
[pid=17316/tid=17323] ppid=17311 vsize=127552 CPUtime=3.05 cores=4-7
/proc/17316/task/17323/stat : 17323 (plingeling) R 17311 17311 16810 0 -1 4202560 1003 1250 0 0 305 0 0 0 25 0 5 0 527167578 130613248 3950 33554432000 4194304 4356188 140737347716640 18446744073709551615 4282486 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17316/tid=17324] ppid=17311 vsize=127552 CPUtime=3.05 cores=4-7
/proc/17316/task/17324/stat : 17324 (plingeling) R 17311 17311 16810 0 -1 4202560 1020 1250 0 0 305 0 0 0 25 0 5 0 527167578 130613248 3950 33554432000 4194304 4356188 140737347716640 18446744073709551615 4238495 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17316/tid=17325] ppid=17311 vsize=127552 CPUtime=3.04 cores=4-7
/proc/17316/task/17325/stat : 17325 (plingeling) R 17311 17311 16810 0 -1 4202560 1022 1250 0 0 304 0 0 0 25 0 5 0 527167578 130613248 3950 33554432000 4194304 4356188 140737347716640 18446744073709551615 4332145 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17316/tid=17326] ppid=17311 vsize=127552 CPUtime=3.03 cores=4-7
/proc/17316/task/17326/stat : 17326 (plingeling) R 17311 17311 16810 0 -1 4202560 1090 1250 0 0 303 0 0 0 25 0 5 0 527167578 130613248 3950 33554432000 4194304 4356188 140737347716640 18446744073709551615 4332239 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.56
Current children cumulated vsize (KiB) 1435328

[startup+6.30115 s]
/proc/loadavg: 8.16 8.15 8.10 9/184 17326
/proc/meminfo: memFree=29922604/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
[pid=17312] ppid=17311 vsize=802856 CPUtime=6.29 cores=0
/proc/17312/stat : 17312 (cryptominisat) R 17311 17311 16810 0 -1 4202496 11894 0 0 0 627 2 0 0 25 0 1 0 527167574 822124544 5946 33554432000 4194304 4587354 140733322103776 18446744073709551615 4388336 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17312/statm: 200714 5946 337 96 0 197276 0
[pid=17313] ppid=17311 vsize=23644 CPUtime=6.28 cores=1
/proc/17313/stat : 17313 (clasp) R 17311 17311 16810 0 -1 4202496 3376 0 0 0 628 0 0 0 25 0 1 0 527167574 24211456 3187 33554432000 4194304 4785740 140733894257840 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17313/statm: 5911 3187 340 145 0 2956 0
[pid=17314] ppid=17311 vsize=458612 CPUtime=6.28 cores=2
/proc/17314/stat : 17314 (TNM) R 17311 17311 16810 0 -1 4202496 1199 0 0 0 628 0 0 0 25 0 1 0 527167574 469618688 1095 33554432000 134512640 134534564 4292253504 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17314/statm: 114653 1095 95 6 0 113319 0
[pid=17315] ppid=17311 vsize=13112 CPUtime=6.29 cores=3
/proc/17315/stat : 17315 (march_hi) R 17311 17311 16810 0 -1 4202496 4802 0 0 0 629 0 0 0 25 0 1 0 527167574 13426688 1476 33554432000 4194304 4331988 140736488535616 18446744073709551615 4210448 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17315/statm: 3278 1476 174 34 0 1720 0
[pid=17316] ppid=17311 vsize=130484 CPUtime=25.01 cores=4-7
/proc/17316/stat : 17316 (plingeling) S 17311 17311 16810 0 -1 4202496 7580 1250 0 0 2500 1 0 0 18 0 5 0 527167574 133615616 4694 33554432000 4194304 4356188 140737347716640 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17316/statm: 32621 4694 162 40 0 31165 0
[pid=17316/tid=17323] ppid=17311 vsize=130484 CPUtime=6.25 cores=4-7
/proc/17316/task/17323/stat : 17323 (plingeling) R 17311 17311 16810 0 -1 4202560 1231 1250 0 0 625 0 0 0 25 0 5 0 527167578 133615616 4694 33554432000 4194304 4356188 140737347716640 18446744073709551615 4271040 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17316/tid=17324] ppid=17311 vsize=130484 CPUtime=6.25 cores=4-7
/proc/17316/task/17324/stat : 17324 (plingeling) R 17311 17311 16810 0 -1 4202560 1246 1250 0 0 625 0 0 0 25 0 5 0 527167578 133615616 4694 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331002 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17316/tid=17325] ppid=17311 vsize=130484 CPUtime=6.24 cores=4-7
/proc/17316/task/17325/stat : 17325 (plingeling) R 17311 17311 16810 0 -1 4202560 1215 1250 0 0 624 0 0 0 25 0 5 0 527167578 133615616 4694 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17316/tid=17326] ppid=17311 vsize=130484 CPUtime=6.22 cores=4-7
/proc/17316/task/17326/stat : 17326 (plingeling) R 17311 17311 16810 0 -1 4202560 1347 1250 0 0 622 0 0 0 25 0 5 0 527167578 133615616 4694 33554432000 4194304 4356188 140737347716640 18446744073709551615 4210575 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.15
Current children cumulated vsize (KiB) 1447032

[startup+12.701 s]
/proc/loadavg: 8.15 8.15 8.10 9/184 17326
/proc/meminfo: memFree=29913924/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
[pid=17312] ppid=17311 vsize=787000 CPUtime=12.69 cores=0
/proc/17312/stat : 17312 (cryptominisat) R 17311 17311 16810 0 -1 4202496 14012 0 0 0 1267 2 0 0 25 0 1 0 527167574 805888000 5422 33554432000 4194304 4587354 140733322103776 18446744073709551615 4238406 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17312/statm: 196750 5422 337 96 0 193312 0
[pid=17313] ppid=17311 vsize=27040 CPUtime=12.68 cores=1
/proc/17313/stat : 17313 (clasp) R 17311 17311 16810 0 -1 4202496 4215 0 0 0 1268 0 0 0 25 0 1 0 527167574 27688960 4026 33554432000 4194304 4785740 140733894257840 18446744073709551615 4286953 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17313/statm: 6760 4026 340 145 0 3805 0
[pid=17314] ppid=17311 vsize=458612 CPUtime=12.68 cores=2
/proc/17314/stat : 17314 (TNM) R 17311 17311 16810 0 -1 4202496 1199 0 0 0 1268 0 0 0 25 0 1 0 527167574 469618688 1095 33554432000 134512640 134534564 4292253504 18446744073709551615 134528504 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17314/statm: 114653 1095 95 6 0 113319 0
[pid=17315] ppid=17311 vsize=14648 CPUtime=12.68 cores=3
/proc/17315/stat : 17315 (march_hi) R 17311 17311 16810 0 -1 4202496 5080 0 0 0 1268 0 0 0 25 0 1 0 527167574 14999552 1754 33554432000 4194304 4331988 140736488535616 18446744073709551615 4299299 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17315/statm: 3662 1754 174 34 0 2104 0
[pid=17316] ppid=17311 vsize=129228 CPUtime=50.58 cores=4-7
/proc/17316/stat : 17316 (plingeling) S 17311 17311 16810 0 -1 4202496 9154 1250 0 0 5056 2 0 0 18 0 5 0 527167574 132329472 4380 33554432000 4194304 4356188 140737347716640 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17316/statm: 32307 4380 162 40 0 30851 0
[pid=17316/tid=17323] ppid=17311 vsize=129228 CPUtime=12.64 cores=4-7
/proc/17316/task/17323/stat : 17323 (plingeling) R 17311 17311 16810 0 -1 4202560 1906 1250 0 0 1264 0 0 0 25 0 5 0 527167578 132329472 4380 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331019 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17316/tid=17324] ppid=17311 vsize=129228 CPUtime=12.65 cores=4-7
/proc/17316/task/17324/stat : 17324 (plingeling) R 17311 17311 16810 0 -1 4202560 1375 1250 0 0 1265 0 0 0 25 0 5 0 527167578 132329472 4380 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17316/tid=17325] ppid=17311 vsize=129228 CPUtime=12.64 cores=4-7
/proc/17316/task/17325/stat : 17325 (plingeling) R 17311 17311 16810 0 -1 4202560 1312 1250 0 0 1264 0 0 0 25 0 5 0 527167578 132329472 4380 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331050 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17316/tid=17326] ppid=17311 vsize=129228 CPUtime=12.58 cores=4-7
/proc/17316/task/17326/stat : 17326 (plingeling) R 17311 17311 16810 0 -1 4202560 2020 1250 0 0 1258 0 0 0 25 0 5 0 527167578 132329472 4380 33554432000 4194304 4356188 140737347716640 18446744073709551615 4279478 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.31
Current children cumulated vsize (KiB) 1434852

[startup+25.5006 s]
/proc/loadavg: 8.12 8.14 8.09 9/184 17326
/proc/meminfo: memFree=29920064/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
[pid=17312] ppid=17311 vsize=785216 CPUtime=25.49 cores=0
/proc/17312/stat : 17312 (cryptominisat) R 17311 17311 16810 0 -1 4202496 23394 0 0 0 2545 4 0 0 25 0 1 0 527167574 804061184 3692 33554432000 4194304 4587354 140733322103776 18446744073709551615 4238593 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=17314] ppid=17311 vsize=458612 CPUtime=4782.13 cores=2
/proc/17314/stat : 17314 (TNM) R 17311 17311 16810 0 -1 4202496 1199 0 0 0 478212 1 0 0 25 0 1 0 527167574 469618688 1095 33554432000 134512640 134534564 4292253504 18446744073709551615 134528988 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17314/statm: 114653 1095 95 6 0 113319 0
[pid=17315] ppid=17311 vsize=694776 CPUtime=4782.43 cores=3
/proc/17315/stat : 17315 (march_hi) R 17311 17311 16810 0 -1 4202496 138675 0 0 0 478147 96 0 0 25 0 1 0 527167574 711450624 135349 33554432000 4194304 4331988 140736488535616 18446744073709551615 4287208 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17315/statm: 173694 135349 175 34 0 172136 0
[pid=17316] ppid=17311 vsize=227732 CPUtime=19102.4 cores=4-7
/proc/17316/stat : 17316 (plingeling) S 17311 17311 16810 0 -1 4202496 455177 1250 0 0 1910170 70 0 0 18 0 5 0 527167574 233197568 29006 33554432000 4194304 4356188 140737347716640 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17316/statm: 56933 29006 162 40 0 55477 0
[pid=17316/tid=17323] ppid=17311 vsize=227732 CPUtime=4782.39 cores=4-7
/proc/17316/task/17323/stat : 17323 (plingeling) R 17311 17311 16810 0 -1 4202560 127391 1250 0 0 478221 18 0 0 25 0 5 0 527167578 233197568 29006 33554432000 4194304 4356188 140737347716640 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17316/tid=17324] ppid=17311 vsize=227732 CPUtime=4782.39 cores=4-7
/proc/17316/task/17324/stat : 17324 (plingeling) R 17311 17311 16810 0 -1 4202560 105380 1250 0 0 478221 18 0 0 25 0 5 0 527167578 233197568 29006 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17316/tid=17325] ppid=17311 vsize=227732 CPUtime=4781.27 cores=4-7
/proc/17316/task/17325/stat : 17325 (plingeling) R 17311 17311 16810 0 -1 4202560 98598 1250 0 0 478110 17 0 0 25 0 5 0 527167578 233197568 29006 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331305 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17316/tid=17326] ppid=17311 vsize=227732 CPUtime=4756.27 cores=4-7
/proc/17316/task/17326/stat : 17326 (plingeling) R 17311 17311 16810 0 -1 4202560 121267 1250 0 0 475612 15 0 0 25 0 5 0 527167578 233197568 29006 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38231.8
Current children cumulated vsize (KiB) 1707200

[startup+4842.3 s]
/proc/loadavg: 8.11 8.13 8.10 9/184 17467
/proc/meminfo: memFree=29121896/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
[pid=17312] ppid=17311 vsize=153984 CPUtime=4842.44 cores=0
/proc/17312/stat : 17312 (cryptominisat) R 17311 17311 16810 0 -1 4202496 989529 0 0 0 483843 401 0 0 25 0 1 0 527167574 157679616 17874 33554432000 4194304 4587354 140733322103776 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17312/statm: 38496 17874 337 96 0 35058 0
[pid=17313] ppid=17311 vsize=155868 CPUtime=4842.43 cores=1
/proc/17313/stat : 17313 (clasp) R 17311 17311 16810 0 -1 4202496 34476 0 0 0 484236 7 0 0 25 0 1 0 527167574 159608832 32883 33554432000 4194304 4785740 140733894257840 18446744073709551615 4287244 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17313/statm: 38967 32883 341 145 0 36012 0
[pid=17314] ppid=17311 vsize=458612 CPUtime=4842.13 cores=2
/proc/17314/stat : 17314 (TNM) R 17311 17311 16810 0 -1 4202496 1199 0 0 0 484212 1 0 0 25 0 1 0 527167574 469618688 1095 33554432000 134512640 134534564 4292253504 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17314/statm: 114653 1095 95 6 0 113319 0
[pid=17315] ppid=17311 vsize=694776 CPUtime=4842.43 cores=3
/proc/17315/stat : 17315 (march_hi) R 17311 17311 16810 0 -1 4202496 140318 0 0 0 484146 97 0 0 25 0 1 0 527167574 711450624 136992 33554432000 4194304 4331988 140736488535616 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17315/statm: 173694 136992 175 34 0 172136 0
[pid=17316] ppid=17311 vsize=217748 CPUtime=19342.1 cores=4-7
/proc/17316/stat : 17316 (plingeling) S 17311 17311 16810 0 -1 4202496 461673 1250 0 0 1934136 71 0 0 18 0 5 0 527167574 222973952 26510 33554432000 4194304 4356188 140737347716640 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17316/statm: 54437 26510 162 40 0 52981 0
[pid=17316/tid=17323] ppid=17311 vsize=217748 CPUtime=4842.39 cores=4-7
/proc/17316/task/17323/stat : 17323 (plingeling) R 17311 17311 16810 0 -1 4202560 129823 1250 0 0 484221 18 0 0 25 0 5 0 527167578 222973952 26510 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17316/tid=17324] ppid=17311 vsize=217748 CPUtime=4842.4 cores=4-7
/proc/17316/task/17324/stat : 17324 (plingeling) R 17311 17311 16810 0 -1 4202560 107812 1250 0 0 484221 19 0 0 25 0 5 0 527167578 222973952 26510 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17316/tid=17325] ppid=17311 vsize=217748 CPUtime=4841.27 cores=4-7
/proc/17316/task/17325/stat : 17325 (plingeling) R 17311 17311 16810 0 -1 4202560 99846 1250 0 0 484110 17 0 0 25 0 5 0 527167578 222973952 26510 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17316/tid=17326] ppid=17311 vsize=217748 CPUtime=4815.94 cores=4-7
/proc/17316/task/17326/stat : 17326 (plingeling) R 17311 17311 16810 0 -1 4202560 121651 1250 0 0 481579 15 0 0 25 0 5 0 527167578 222973952 26510 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331355 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38711.5
Current children cumulated vsize (KiB) 1699312

[startup+4902.3 s]
/proc/loadavg: 8.19 8.15 8.11 9/184 17468
/proc/meminfo: memFree=29122236/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
[pid=17312] ppid=17311 vsize=825792 CPUtime=4902.44 cores=0
/proc/17312/stat : 17312 (cryptominisat) R 17311 17311 16810 0 -1 4202496 998553 0 0 0 489838 406 0 0 25 0 1 0 527167574 845611008 15449 33554432000 4194304 4587354 140733322103776 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17312/statm: 206448 15449 337 96 0 203010 0
[pid=17313] ppid=17311 vsize=155868 CPUtime=4902.43 cores=1
/proc/17313/stat : 17313 (clasp) R 17311 17311 16810 0 -1 4202496 34476 0 0 0 490236 7 0 0 25 0 1 0 527167574 159608832 32883 33554432000 4194304 4785740 140733894257840 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17313/statm: 38967 32883 341 145 0 36012 0
[pid=17314] ppid=17311 vsize=458612 CPUtime=4902.13 cores=2
/proc/17314/stat : 17314 (TNM) R 17311 17311 16810 0 -1 4202496 1199 0 0 0 490212 1 0 0 25 0 1 0 527167574 469618688 1095 33554432000 134512640 134534564 4292253504 18446744073709551615 134530522 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17314/statm: 114653 1095 95 6 0 113319 0
[pid=17315] ppid=17311 vsize=694776 CPUtime=4902.44 cores=3
/proc/17315/stat : 17315 (march_hi) R 17311 17311 16810 0 -1 4202496 141967 0 0 0 490145 99 0 0 25 0 1 0 527167574 711450624 138641 33554432000 4194304 4331988 140736488535616 18446744073709551615 4299243 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17315/statm: 173694 138641 175 34 0 172136 0
[pid=17316] ppid=17311 vsize=220948 CPUtime=19581.8 cores=4-7
/proc/17316/stat : 17316 (plingeling) S 17311 17311 16810 0 -1 4202496 466153 1250 0 0 1958103 72 0 0 18 0 5 0 527167574 226250752 27310 33554432000 4194304 4356188 140737347716640 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17316/statm: 55237 27310 162 40 0 53781 0
[pid=17316/tid=17323] ppid=17311 vsize=220948 CPUtime=4902.39 cores=4-7
/proc/17316/task/17323/stat : 17323 (plingeling) R 17311 17311 16810 0 -1 4202560 129823 1250 0 0 490221 18 0 0 25 0 5 0 527167578 226250752 27310 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331337 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17316/tid=17324] ppid=17311 vsize=220948 CPUtime=4902.4 cores=4-7
/proc/17316/task/17324/stat : 17324 (plingeling) R 17311 17311 16810 0 -1 4202560 107812 1250 0 0 490221 19 0 0 25 0 5 0 527167578 226250752 27310 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17316/tid=17325] ppid=17311 vsize=220948 CPUtime=4901.26 cores=4-7
/proc/17316/task/17325/stat : 17325 (plingeling) R 17311 17311 16810 0 -1 4202560 102278 1250 0 0 490109 17 0 0 25 0 5 0 527167578 226250752 27310 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331984 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17316/tid=17326] ppid=17311 vsize=220948 CPUtime=4875.62 cores=4-7
/proc/17316/task/17326/stat : 17326 (plingeling) R 17311 17311 16810 0 -1 4202560 123699 1250 0 0 487547 15 0 0 25 0 5 0 527167578 226250752 27310 33554432000 4194304 4356188 140737347716640 18446744073709551615 4210579 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39191.2
Current children cumulated vsize (KiB) 2374320

[startup+4962.3 s]
/proc/loadavg: 8.07 8.12 8.09 9/184 17470
/proc/meminfo: memFree=29084016/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
[pid=17312] ppid=17311 vsize=75528 CPUtime=4962.44 cores=0
/proc/17312/stat : 17312 (cryptominisat) R 17311 17311 16810 0 -1 4202496 1006966 0 0 0 495834 410 0 0 25 0 1 0 527167574 77340672 12350 33554432000 4194304 4587354 140733322103776 18446744073709551615 4238463 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17312/statm: 18882 12350 337 96 0 15444 0
[pid=17313] ppid=17311 vsize=156292 CPUtime=4962.43 cores=1
/proc/17313/stat : 17313 (clasp) R 17311 17311 16810 0 -1 4202496 34581 0 0 0 496236 7 0 0 25 0 1 0 527167574 160043008 32988 33554432000 4194304 4785740 140733894257840 18446744073709551615 4287129 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17313/statm: 39073 32988 341 145 0 36118 0
[pid=17314] ppid=17311 vsize=458612 CPUtime=4962.13 cores=2
/proc/17314/stat : 17314 (TNM) R 17311 17311 16810 0 -1 4202496 1199 0 0 0 496212 1 0 0 25 0 1 0 527167574 469618688 1095 33554432000 134512640 134534564 4292253504 18446744073709551615 134528824 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17314/statm: 114653 1095 95 6 0 113319 0
[pid=17315] ppid=17311 vsize=694776 CPUtime=4962.44 cores=3
/proc/17315/stat : 17315 (march_hi) R 17311 17311 16810 0 -1 4202496 143627 0 0 0 496144 100 0 0 25 0 1 0 527167574 711450624 140301 33554432000 4194304 4331988 140736488535616 18446744073709551615 4210459 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17315/statm: 173694 140301 175 34 0 172136 0
[pid=17316] ppid=17311 vsize=227732 CPUtime=19821.4 cores=4-7
/proc/17316/stat : 17316 (plingeling) S 17311 17311 16810 0 -1 4202496 469897 1250 0 0 1982070 72 0 0 18 0 5 0 527167574 233197568 29006 33554432000 4194304 4356188 140737347716640 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17316/statm: 56933 29006 162 40 0 55477 0
[pid=17316/tid=17323] ppid=17311 vsize=227732 CPUtime=4962.39 cores=4-7
/proc/17316/task/17323/stat : 17323 (plingeling) R 17311 17311 16810 0 -1 4202560 131071 1250 0 0 496221 18 0 0 25 0 5 0 527167578 233197568 29006 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17316/tid=17324] ppid=17311 vsize=227732 CPUtime=4962.4 cores=4-7
/proc/17316/task/17324/stat : 17324 (plingeling) R 17311 17311 16810 0 -1 4202560 109060 1250 0 0 496221 19 0 0 25 0 5 0 527167578 233197568 29006 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331964 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17316/tid=17325] ppid=17311 vsize=227732 CPUtime=4961.25 cores=4-7
/proc/17316/task/17325/stat : 17325 (plingeling) R 17311 17311 16810 0 -1 4202560 102278 1250 0 0 496108 17 0 0 25 0 5 0 527167578 233197568 29006 33554432000 4194304 4356188 140737347716640 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17316/tid=17326] ppid=17311 vsize=227732 CPUtime=4935.3 cores=4-7
/proc/17316/task/17326/stat : 17326 (plingeling) R 17311 17311 16810 0 -1 4202560 124947 1250 0 0 493514 16 0 0 25 0 5 0 527167578 233197568 29006 33554432000 4194304 4356188 140737347716640 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39670.9
Current children cumulated vsize (KiB) 1631264



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.13 8.14 8.10 9/184 17471
/proc/meminfo: memFree=29113872/32951124 swapFree=67102748/67111528
[pid=17311] ppid=17309 vsize=18324 CPUtime=0 cores=0-7
/proc/17311/stat : 17311 (ppfolio) S 17309 17311 16810 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 527167573 18763776 390 33554432000 4194304 4287625 140735412569104 18446744073709551615 246516854287 0 0 4096 65536 4559280000007667711 0 0 17 4 0 0 0
/proc/17311/statm: 4581 390 341 23 0 91 0
[pid=17312] ppid=17311 vsize=154900 CPUtime=5000.15 cores=0
/proc/17312/stat : 17312 (cryptominisat) R 17311 17311 16810 0 -1 4202496 1012172 0 0 0 499601 414 0 0 25 0 1 0 527167574 158617600 17490 33554432000 4194304 4587354 140733322103776 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17312/statm: 38725 17490 337 96 0 35287 0
[pid=17313] ppid=17311 vsize=157148 CPUtime=5000.14 cores=1
/proc/17313/stat : 17313 (clasp) R 17311 17311 16810 0 -1 4202496 34794 0 0 0 500007 7 0 0 25 0 1 0 527167574 160919552 33201 33554432000 4194304 4785740 140733894257840 18446744073709551615 4278129 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17313/statm: 39287 33201 341 145 0 36332 0
[pid=17314] ppid=17311 vsize=458612 CPUtime=4999.83 cores=2
/proc/17314/stat : 17314 (TNM) R 17311 17311 16810 0 -1 4202496 1199 0 0 0 499982 1 0 0 25 0 1 0 527167574 469618688 1095 33554432000 134512640 134534564 4292253504 18446744073709551615 134528504 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/17314/statm: 114653 1095 95 6 0 113319 0
[pid=17315] ppid=17311 vsize=694776 CPUtime=5000.14 cores=3
/proc/17315/stat : 17315 (march_hi) R 17311 17311 16810 0 -1 4202496 144670 0 0 0 499914 100 0 0 25 0 1 0 527167574 711450624 141344 33554432000 4194304 4331988 140736488535616 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17315/statm: 173694 141344 175 34 0 172136 0
[pid=17316] ppid=17311 vsize=209172 CPUtime=19972 cores=4-7
/proc/17316/stat : 17316 (plingeling) S 17311 17311 16810 0 -1 4202496 470985 1250 0 0 1997131 72 0 0 18 0 5 0 527167574 214192128 24366 33554432000 4194304 4356188 140737347716640 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/17316/statm: 52293 24366 162 40 0 50837 0
[pid=17316/tid=17323] ppid=17311 vsize=209172 CPUtime=5000.1 cores=4-7
/proc/17316/task/17323/stat : 17323 (plingeling) R 17311 17311 16810 0 -1 4202560 131455 1250 0 0 499992 18 0 0 25 0 5 0 527167578 214192128 24366 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331792 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=17316/tid=17324] ppid=17311 vsize=209172 CPUtime=5000.11 cores=4-7
/proc/17316/task/17324/stat : 17324 (plingeling) R 17311 17311 16810 0 -1 4202560 109444 1250 0 0 499992 19 0 0 25 0 5 0 527167578 214192128 24366 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331188 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17316/tid=17325] ppid=17311 vsize=209172 CPUtime=4998.96 cores=4-7
/proc/17316/task/17325/stat : 17325 (plingeling) R 17311 17311 16810 0 -1 4202560 102470 1250 0 0 499879 17 0 0 25 0 5 0 527167578 214192128 24366 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17316/tid=17326] ppid=17311 vsize=209172 CPUtime=4972.8 cores=4-7
/proc/17316/task/17326/stat : 17326 (plingeling) R 17311 17311 16810 0 -1 4202560 125075 1250 0 0 497264 16 0 0 25 0 5 0 527167578 214192128 24366 33554432000 4194304 4356188 140737347716640 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39972.3
Current children cumulated vsize (KiB) 1692932

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 17311 and gives
#  childrusage.ru_utime.tv_sec=14995
#  childrusage.ru_utime.tv_usec=927272
#  childrusage.ru_stime.tv_sec=4
#  childrusage.ru_stime.tv_usec=261352
# CPU time returned by wait4() is 15000.2
# while last known CPU time is 39972.3
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39972.3
CPU user time (s): 39966.4
CPU system time (s): 5.94
CPU usage (%): 799.443
Max. virtual memory (cumulated for all children) (KiB): 7449584

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14995.9
system time used= 4.26135
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1048702
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 718
involuntary context switches= 33317

runsolver used 7.69183 second user time and 19.588 second system time

The end

Launcher Data 

Begin job on node134 at 2011-04-30 11:28:14
IDJOB=3220071
IDBENCH=82695
IDSOLVER=1638
FILE ID=node134/3220071-1304155694
RUNJOBID= node134-1304145462-16824
PBS_JOBID= 13170886
Free space on /tmp= 72512 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k7-r85-v250-c21250-S977260632-016-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3220071-1304155694/watcher-3220071-1304155694 -o /tmp/evaluation-result-3220071-1304155694/solver-3220071-1304155694 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3220071-1304155694.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 58eb79e3bfc8d25580f794d4e2d18629
RANDOM SEED=1349988148

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29981812 kB
Buffers:       1028800 kB
Cached:        1480444 kB
SwapCached:       4600 kB
Active:         211664 kB
Inactive:      2357300 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29981812 kB
SwapTotal:    67111528 kB
SwapFree:     67102748 kB
Dirty:           10512 kB
Writeback:           8 kB
AnonPages:       56864 kB
Mapped:          14608 kB
Slab:           335416 kB
PageTables:       3892 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   186124 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72500 MiB
End job on node134 at 2011-04-30 12:51:36