Trace number 3220047

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39972 5000.02

General information on the benchmark

Namerandom/large/
unif-k7-r85-v400-c34000-S94779963-093-UNKNOWN.cnf
MD5SUMaef8bd4e13046b42a5bc777dc56e10de
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables400
Number of clauses34000
Sum of the clauses size238000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 534000

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3220047-1304155579/watcher-3220047-1304155579 -o /tmp/evaluation-result-3220047-1304155579/solver-3220047-1304155579 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3220047-1304155579.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.24 8.13 8.03 2/170 30499
/proc/meminfo: memFree=30980032/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) R 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 25 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0

[startup+0.0991051 s]
/proc/loadavg: 8.24 8.13 8.03 2/170 30499
/proc/meminfo: memFree=30980032/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 25 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.101097 s]
/proc/loadavg: 8.24 8.13 8.03 2/170 30499
/proc/meminfo: memFree=30980032/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 25 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.301035 s]
/proc/loadavg: 8.24 8.13 8.03 2/170 30499
/proc/meminfo: memFree=30980032/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 25 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.70096 s]
/proc/loadavg: 8.24 8.13 8.03 2/170 30499
/proc/meminfo: memFree=30980032/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 25 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50082 s]
/proc/loadavg: 8.22 8.12 8.03 9/180 30514
/proc/meminfo: memFree=30927624/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
[pid=30500] ppid=30499 vsize=448496 CPUtime=1.49 cores=0
/proc/30500/stat : 30500 (cryptominisat) R 30499 30499 30227 0 -1 4202496 6289 0 0 0 149 0 0 0 20 0 1 0 507974140 459259904 2067 33554432000 4194304 4587354 140734713304736 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30500/statm: 112124 2067 327 96 0 108686 0
[pid=30501] ppid=30499 vsize=24408 CPUtime=1.48 cores=1
/proc/30501/stat : 30501 (clasp) R 30499 30499 30227 0 -1 4202496 3694 0 0 0 148 0 0 0 20 0 1 0 507974140 24993792 3426 33554432000 4194304 4785740 140735892711152 18446744073709551615 4550931 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30501/statm: 6102 3426 340 145 0 3147 0
[pid=30502] ppid=30499 vsize=456220 CPUtime=1.49 cores=2
/proc/30502/stat : 30502 (TNM) R 30499 30499 30227 0 -1 4202496 1457 0 0 0 149 0 0 0 20 0 1 0 507974140 467169280 1354 33554432000 134512640 134534564 4292144256 18446744073709551615 134524577 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30502/statm: 114055 1354 91 6 0 112721 0
[pid=30503] ppid=30499 vsize=12556 CPUtime=1.48 cores=3
/proc/30503/stat : 30503 (march_hi) R 30499 30499 30227 0 -1 4202496 6381 0 0 0 148 0 0 0 20 0 1 0 507974140 12857344 1361 33554432000 4194304 4331988 140734841743984 18446744073709551615 4271154 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30503/statm: 3139 1361 150 34 0 1581 0
[pid=30504] ppid=30499 vsize=128048 CPUtime=5.71 cores=4-7
/proc/30504/stat : 30504 (plingeling) S 30499 30499 30227 0 -1 4202496 7506 1256 0 0 571 0 0 0 18 0 5 0 507974140 131121152 4052 33554432000 4194304 4356188 140733478187568 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/30504/statm: 32012 4052 161 40 0 30556 0
[pid=30504/tid=30511] ppid=30499 vsize=128048 CPUtime=1.42 cores=4-7
/proc/30504/task/30511/stat : 30511 (plingeling) R 30499 30499 30227 0 -1 4202560 586 1256 0 0 142 0 0 0 25 0 5 0 507974147 131121152 4052 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331964 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30504/tid=30512] ppid=30499 vsize=128048 CPUtime=1.39 cores=4-7
/proc/30504/task/30512/stat : 30512 (plingeling) R 30499 30499 30227 0 -1 4202560 587 1256 0 0 139 0 0 0 25 0 5 0 507974147 131121152 4052 33554432000 4194304 4356188 140733478187568 18446744073709551615 4332165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30504/tid=30513] ppid=30499 vsize=128048 CPUtime=1.41 cores=4-7
/proc/30504/task/30513/stat : 30513 (plingeling) R 30499 30499 30227 0 -1 4202560 584 1256 0 0 141 0 0 0 23 0 5 0 507974147 131121152 4052 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=30504/tid=30514] ppid=30499 vsize=128048 CPUtime=1.42 cores=4-7
/proc/30504/task/30514/stat : 30514 (plingeling) R 30499 30499 30227 0 -1 4202560 1125 1256 0 0 142 0 0 0 23 0 5 0 507974147 131121152 4052 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.65
Current children cumulated vsize (KiB) 1088052

[startup+3.10053 s]
/proc/loadavg: 8.22 8.12 8.03 9/180 30514
/proc/meminfo: memFree=30931592/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
[pid=30500] ppid=30499 vsize=452128 CPUtime=3.09 cores=0
/proc/30500/stat : 30500 (cryptominisat) R 30499 30499 30227 0 -1 4202496 9456 0 0 0 308 1 0 0 25 0 1 0 507974140 462979072 2767 33554432000 4194304 4587354 140734713304736 18446744073709551615 4238418 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30500/statm: 113032 2767 327 96 0 109594 0
[pid=30501] ppid=30499 vsize=25604 CPUtime=3.08 cores=1
/proc/30501/stat : 30501 (clasp) R 30499 30499 30227 0 -1 4202496 3988 0 0 0 308 0 0 0 25 0 1 0 507974140 26218496 3720 33554432000 4194304 4785740 140735892711152 18446744073709551615 4287038 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30501/statm: 6401 3720 340 145 0 3446 0
[pid=30502] ppid=30499 vsize=456220 CPUtime=3.09 cores=2
/proc/30502/stat : 30502 (TNM) R 30499 30499 30227 0 -1 4202496 1457 0 0 0 309 0 0 0 25 0 1 0 507974140 467169280 1354 33554432000 134512640 134534564 4292144256 18446744073709551615 134524573 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30502/statm: 114055 1354 91 6 0 112721 0
[pid=30503] ppid=30499 vsize=17024 CPUtime=3.08 cores=3
/proc/30503/stat : 30503 (march_hi) R 30499 30499 30227 0 -1 4202496 7763 0 0 0 308 0 0 0 25 0 1 0 507974140 17432576 2243 33554432000 4194304 4331988 140734841743984 18446744073709551615 4308807 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30503/statm: 4256 2243 174 34 0 2698 0
[pid=30504] ppid=30499 vsize=127940 CPUtime=12.11 cores=4-7
/proc/30504/stat : 30504 (plingeling) S 30499 30499 30227 0 -1 4202496 9527 1256 0 0 1210 1 0 0 18 0 5 0 507974140 131010560 4025 33554432000 4194304 4356188 140733478187568 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/30504/statm: 31985 4025 161 40 0 30529 0
[pid=30504/tid=30511] ppid=30499 vsize=127940 CPUtime=3.02 cores=4-7
/proc/30504/task/30511/stat : 30511 (plingeling) R 30499 30499 30227 0 -1 4202560 1163 1256 0 0 302 0 0 0 25 0 5 0 507974147 131010560 4025 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331076 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30504/tid=30512] ppid=30499 vsize=127940 CPUtime=2.98 cores=4-7
/proc/30504/task/30512/stat : 30512 (plingeling) R 30499 30499 30227 0 -1 4202560 1204 1256 0 0 298 0 0 0 25 0 5 0 507974147 131010560 4025 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30504/tid=30513] ppid=30499 vsize=127940 CPUtime=3.01 cores=4-7
/proc/30504/task/30513/stat : 30513 (plingeling) R 30499 30499 30227 0 -1 4202560 1234 1256 0 0 301 0 0 0 25 0 5 0 507974147 131010560 4025 33554432000 4194304 4356188 140733478187568 18446744073709551615 4332023 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=30504/tid=30514] ppid=30499 vsize=127940 CPUtime=3.02 cores=4-7
/proc/30504/task/30514/stat : 30514 (plingeling) R 30499 30499 30227 0 -1 4202560 1302 1256 0 0 302 0 0 0 25 0 5 0 507974147 131010560 4025 33554432000 4194304 4356188 140733478187568 18446744073709551615 4234139 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.45
Current children cumulated vsize (KiB) 1097240

[startup+6.30096 s]
/proc/loadavg: 8.20 8.12 8.03 9/180 30514
/proc/meminfo: memFree=30905904/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
[pid=30500] ppid=30499 vsize=470148 CPUtime=6.29 cores=0
/proc/30500/stat : 30500 (cryptominisat) R 30499 30499 30227 0 -1 4202496 15240 0 0 0 627 2 0 0 25 0 1 0 507974140 481431552 7093 33554432000 4194304 4587354 140734713304736 18446744073709551615 4238481 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30500/statm: 117537 7093 337 96 0 114099 0
[pid=30501] ppid=30499 vsize=28160 CPUtime=6.28 cores=1
/proc/30501/stat : 30501 (clasp) R 30499 30499 30227 0 -1 4202496 4606 0 0 0 628 0 0 0 25 0 1 0 507974140 28835840 4338 33554432000 4194304 4785740 140735892711152 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30501/statm: 7040 4338 340 145 0 4085 0
[pid=30502] ppid=30499 vsize=460708 CPUtime=6.29 cores=2
/proc/30502/stat : 30502 (TNM) R 30499 30499 30227 0 -1 4202496 1815 0 0 0 629 0 0 0 25 0 1 0 507974140 471764992 1712 33554432000 134512640 134534564 4292144256 18446744073709551615 134529777 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30502/statm: 115177 1712 95 6 0 113843 0
[pid=30503] ppid=30499 vsize=17024 CPUtime=6.28 cores=3
/proc/30503/stat : 30503 (march_hi) R 30499 30499 30227 0 -1 4202496 7763 0 0 0 628 0 0 0 25 0 1 0 507974140 17432576 2243 33554432000 4194304 4331988 140734841743984 18446744073709551615 4265440 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30503/statm: 4256 2243 174 34 0 2698 0
[pid=30504] ppid=30499 vsize=132776 CPUtime=24.89 cores=4-7
/proc/30504/stat : 30504 (plingeling) S 30499 30499 30227 0 -1 4202496 11791 1256 0 0 2488 1 0 0 18 0 5 0 507974140 135962624 5265 33554432000 4194304 4356188 140733478187568 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/30504/statm: 33194 5265 161 40 0 31738 0
[pid=30504/tid=30511] ppid=30499 vsize=132776 CPUtime=6.22 cores=4-7
/proc/30504/task/30511/stat : 30511 (plingeling) R 30499 30499 30227 0 -1 4202560 1931 1256 0 0 622 0 0 0 25 0 5 0 507974147 135962624 5265 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30504/tid=30512] ppid=30499 vsize=132776 CPUtime=6.16 cores=4-7
/proc/30504/task/30512/stat : 30512 (plingeling) R 30499 30499 30227 0 -1 4202560 1451 1256 0 0 616 0 0 0 25 0 5 0 507974147 135962624 5265 33554432000 4194304 4356188 140733478187568 18446744073709551615 4332239 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30504/tid=30513] ppid=30499 vsize=132776 CPUtime=6.21 cores=4-7
/proc/30504/task/30513/stat : 30513 (plingeling) R 30499 30499 30227 0 -1 4202560 1298 1256 0 0 621 0 0 0 25 0 5 0 507974147 135962624 5265 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=30504/tid=30514] ppid=30499 vsize=132776 CPUtime=6.22 cores=4-7
/proc/30504/task/30514/stat : 30514 (plingeling) R 30499 30499 30227 0 -1 4202560 2487 1256 0 0 622 0 0 0 25 0 5 0 507974147 135962624 5265 33554432000 4194304 4356188 140733478187568 18446744073709551615 4210579 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.03
Current children cumulated vsize (KiB) 1127140

[startup+12.7008 s]
/proc/loadavg: 8.18 8.12 8.03 9/180 30514
/proc/meminfo: memFree=30872604/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
[pid=30500] ppid=30499 vsize=487140 CPUtime=12.69 cores=0
/proc/30500/stat : 30500 (cryptominisat) R 30499 30499 30227 0 -1 4202496 21943 0 0 0 1266 3 0 0 25 0 1 0 507974140 498831360 12218 33554432000 4194304 4587354 140734713304736 18446744073709551615 4364908 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30500/statm: 121785 12218 337 96 0 118347 0
[pid=30501] ppid=30499 vsize=32576 CPUtime=12.69 cores=1
/proc/30501/stat : 30501 (clasp) R 30499 30499 30227 0 -1 4202496 5667 0 0 0 1268 1 0 0 25 0 1 0 507974140 33357824 5399 33554432000 4194304 4785740 140735892711152 18446744073709551615 4287008 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30501/statm: 8144 5399 340 145 0 5189 0
[pid=30502] ppid=30499 vsize=460708 CPUtime=12.69 cores=2
/proc/30502/stat : 30502 (TNM) R 30499 30499 30227 0 -1 4202496 1815 0 0 0 1269 0 0 0 25 0 1 0 507974140 471764992 1712 33554432000 134512640 134534564 4292144256 18446744073709551615 134528483 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30502/statm: 115177 1712 95 6 0 113843 0
[pid=30503] ppid=30499 vsize=17024 CPUtime=12.68 cores=3
/proc/30503/stat : 30503 (march_hi) R 30499 30499 30227 0 -1 4202496 7763 0 0 0 1268 0 0 0 25 0 1 0 507974140 17432576 2243 33554432000 4194304 4331988 140734841743984 18446744073709551615 4246566 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30503/statm: 4256 2243 174 34 0 2698 0
[pid=30504] ppid=30499 vsize=136088 CPUtime=50.46 cores=4-7
/proc/30504/stat : 30504 (plingeling) S 30499 30499 30227 0 -1 4202496 15916 1256 0 0 5044 2 0 0 18 0 5 0 507974140 139354112 6094 33554432000 4194304 4356188 140733478187568 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/30504/statm: 34022 6094 161 40 0 32566 0
[pid=30504/tid=30511] ppid=30499 vsize=136088 CPUtime=12.62 cores=4-7
/proc/30504/task/30511/stat : 30511 (plingeling) R 30499 30499 30227 0 -1 4202560 3396 1256 0 0 1262 0 0 0 25 0 5 0 507974147 139354112 6094 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30504/tid=30512] ppid=30499 vsize=136088 CPUtime=12.53 cores=4-7
/proc/30504/task/30512/stat : 30512 (plingeling) R 30499 30499 30227 0 -1 4202560 2862 1256 0 0 1253 0 0 0 25 0 5 0 507974147 139354112 6094 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30504/tid=30513] ppid=30499 vsize=136088 CPUtime=12.61 cores=4-7
/proc/30504/task/30513/stat : 30513 (plingeling) R 30499 30499 30227 0 -1 4202560 1938 1256 0 0 1261 0 0 0 25 0 5 0 507974147 139354112 6094 33554432000 4194304 4356188 140733478187568 18446744073709551615 4238813 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=30504/tid=30514] ppid=30499 vsize=136088 CPUtime=12.61 cores=4-7
/proc/30504/task/30514/stat : 30514 (plingeling) R 30499 30499 30227 0 -1 4202560 3096 1256 0 0 1261 0 0 0 25 0 5 0 507974147 139354112 6094 33554432000 4194304 4356188 140733478187568 18446744073709551615 4332136 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.21
Current children cumulated vsize (KiB) 1151860

[startup+25.5005 s]
/proc/loadavg: 8.14 8.11 8.03 9/180 30515
/proc/meminfo: memFree=30860428/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
[pid=30500] ppid=30499 vsize=491492 CPUtime=25.49 cores=0
/proc/30500/stat : 30500 (cryptominisat) R 30499 30499 30227 0 -1 4202496 31738 0 0 0 2544 5 0 0 25 0 1 0 507974140 503287808 13250 33554432000 4194304 4587354 140734713304736 18446744073709551615 4364793 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=30502] ppid=30499 vsize=460708 CPUtime=4782.25 cores=2
/proc/30502/stat : 30502 (TNM) R 30499 30499 30227 0 -1 4202496 1815 0 0 0 478223 2 0 0 25 0 1 0 507974140 471764992 1712 33554432000 134512640 134534564 4292144256 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30502/statm: 115177 1712 95 6 0 113843 0
[pid=30503] ppid=30499 vsize=472456 CPUtime=4782.41 cores=3
/proc/30503/stat : 30503 (march_hi) R 30499 30499 30227 0 -1 4202496 119116 0 0 0 478160 81 0 0 25 0 1 0 507974140 483794944 113596 33554432000 4194304 4331988 140734841743984 18446744073709551615 4287610 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30503/statm: 118114 113596 175 34 0 116556 0
[pid=30504] ppid=30499 vsize=222216 CPUtime=19102.1 cores=4-7
/proc/30504/stat : 30504 (plingeling) S 30499 30499 30227 0 -1 4202496 384715 1256 0 0 1910146 62 0 0 18 0 5 0 507974140 227549184 27625 33554432000 4194304 4356188 140733478187568 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/30504/statm: 55554 27625 161 40 0 54098 0
[pid=30504/tid=30511] ppid=30499 vsize=222216 CPUtime=4782.34 cores=4-7
/proc/30504/task/30511/stat : 30511 (plingeling) R 30499 30499 30227 0 -1 4202560 105036 1256 0 0 478217 17 0 0 25 0 5 0 507974147 227549184 27625 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30504/tid=30512] ppid=30499 vsize=222216 CPUtime=4755.42 cores=4-7
/proc/30504/task/30512/stat : 30512 (plingeling) R 30499 30499 30227 0 -1 4202560 80084 1256 0 0 475531 11 0 0 25 0 5 0 507974147 227549184 27625 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30504/tid=30513] ppid=30499 vsize=222216 CPUtime=4782.33 cores=4-7
/proc/30504/task/30513/stat : 30513 (plingeling) R 30499 30499 30227 0 -1 4202560 81882 1256 0 0 478219 14 0 0 25 0 5 0 507974147 227549184 27625 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331749 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=30504/tid=30514] ppid=30499 vsize=222216 CPUtime=4781.88 cores=4-7
/proc/30504/task/30514/stat : 30514 (plingeling) R 30499 30499 30227 0 -1 4202560 113089 1256 0 0 478171 17 0 0 25 0 5 0 507974147 227549184 27625 33554432000 4194304 4356188 140733478187568 18446744073709551615 4279688 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38231.6
Current children cumulated vsize (KiB) 2051108

[startup+4842.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/180 30655
/proc/meminfo: memFree=30031640/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
[pid=30500] ppid=30499 vsize=684504 CPUtime=4842.42 cores=0
/proc/30500/stat : 30500 (cryptominisat) R 30499 30499 30227 0 -1 4202496 1134721 0 0 0 483766 476 0 0 25 0 1 0 507974140 700932096 53306 33554432000 4194304 4587354 140734713304736 18446744073709551615 4388345 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30500/statm: 171126 53306 337 96 0 167688 0
[pid=30501] ppid=30499 vsize=192900 CPUtime=4842.41 cores=1
/proc/30501/stat : 30501 (clasp) R 30499 30499 30227 0 -1 4202496 42686 0 0 0 484232 9 0 0 25 0 1 0 507974140 197529600 41451 33554432000 4194304 4785740 140735892711152 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30501/statm: 48225 41451 341 145 0 45270 0
[pid=30502] ppid=30499 vsize=460708 CPUtime=4842.25 cores=2
/proc/30502/stat : 30502 (TNM) R 30499 30499 30227 0 -1 4202496 1815 0 0 0 484223 2 0 0 25 0 1 0 507974140 471764992 1712 33554432000 134512640 134534564 4292144256 18446744073709551615 134530635 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30502/statm: 115177 1712 95 6 0 113843 0
[pid=30503] ppid=30499 vsize=472456 CPUtime=4842.41 cores=3
/proc/30503/stat : 30503 (march_hi) R 30499 30499 30227 0 -1 4202496 120589 0 0 0 484159 82 0 0 25 0 1 0 507974140 483794944 115069 33554432000 4194304 4331988 140734841743984 18446744073709551615 4239530 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30503/statm: 118114 115069 175 34 0 116556 0
[pid=30504] ppid=30499 vsize=216072 CPUtime=19341.8 cores=4-7
/proc/30504/stat : 30504 (plingeling) S 30499 30499 30227 0 -1 4202496 385227 1256 0 0 1934113 62 0 0 18 0 5 0 507974140 221257728 26089 33554432000 4194304 4356188 140733478187568 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/30504/statm: 54018 26089 161 40 0 52562 0
[pid=30504/tid=30511] ppid=30499 vsize=216072 CPUtime=4842.35 cores=4-7
/proc/30504/task/30511/stat : 30511 (plingeling) R 30499 30499 30227 0 -1 4202560 105036 1256 0 0 484218 17 0 0 25 0 5 0 507974147 221257728 26089 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30504/tid=30512] ppid=30499 vsize=216072 CPUtime=4815.09 cores=4-7
/proc/30504/task/30512/stat : 30512 (plingeling) R 30499 30499 30227 0 -1 4202560 80404 1256 0 0 481498 11 0 0 25 0 5 0 507974147 221257728 26089 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30504/tid=30513] ppid=30499 vsize=216072 CPUtime=4842.33 cores=4-7
/proc/30504/task/30513/stat : 30513 (plingeling) R 30499 30499 30227 0 -1 4202560 82074 1256 0 0 484219 14 0 0 25 0 5 0 507974147 221257728 26089 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331062 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=30504/tid=30514] ppid=30499 vsize=216072 CPUtime=4841.88 cores=4-7
/proc/30504/task/30514/stat : 30514 (plingeling) R 30499 30499 30227 0 -1 4202560 113089 1256 0 0 484171 17 0 0 25 0 5 0 507974147 221257728 26089 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38711.2
Current children cumulated vsize (KiB) 2044964

[startup+4902.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/180 30657
/proc/meminfo: memFree=30156948/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
[pid=30500] ppid=30499 vsize=524860 CPUtime=4902.41 cores=0
/proc/30500/stat : 30500 (cryptominisat) R 30499 30499 30227 0 -1 4202496 1134721 0 0 0 489765 476 0 0 25 0 1 0 507974140 537456640 16353 33554432000 4194304 4587354 140734713304736 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30500/statm: 131215 16353 337 96 0 127777 0
[pid=30501] ppid=30499 vsize=193760 CPUtime=4902.42 cores=1
/proc/30501/stat : 30501 (clasp) R 30499 30499 30227 0 -1 4202496 42929 0 0 0 490233 9 0 0 25 0 1 0 507974140 198410240 41587 33554432000 4194304 4785740 140735892711152 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30501/statm: 48440 41587 341 145 0 45485 0
[pid=30502] ppid=30499 vsize=460708 CPUtime=4902.25 cores=2
/proc/30502/stat : 30502 (TNM) R 30499 30499 30227 0 -1 4202496 1815 0 0 0 490223 2 0 0 25 0 1 0 507974140 471764992 1712 33554432000 134512640 134534564 4292144256 18446744073709551615 134528708 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30502/statm: 115177 1712 95 6 0 113843 0
[pid=30503] ppid=30499 vsize=696220 CPUtime=4902.42 cores=3
/proc/30503/stat : 30503 (march_hi) R 30499 30499 30227 0 -1 4202496 121949 0 0 0 490158 84 0 0 25 0 1 0 507974140 712929280 116429 33554432000 4194304 4331988 140734841743984 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30503/statm: 174055 116429 175 34 0 172497 0
[pid=30504] ppid=30499 vsize=231688 CPUtime=19581.4 cores=4-7
/proc/30504/stat : 30504 (plingeling) S 30499 30499 30227 0 -1 4202496 393995 1256 0 0 1958080 63 0 0 18 0 5 0 507974140 237248512 29993 33554432000 4194304 4356188 140733478187568 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/30504/statm: 57922 29993 161 40 0 56466 0
[pid=30504/tid=30511] ppid=30499 vsize=231688 CPUtime=4902.35 cores=4-7
/proc/30504/task/30511/stat : 30511 (plingeling) R 30499 30499 30227 0 -1 4202560 105228 1256 0 0 490218 17 0 0 25 0 5 0 507974147 237248512 29993 33554432000 4194304 4356188 140733478187568 18446744073709551615 4332136 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30504/tid=30512] ppid=30499 vsize=231688 CPUtime=4874.77 cores=4-7
/proc/30504/task/30512/stat : 30512 (plingeling) R 30499 30499 30227 0 -1 4202560 82452 1256 0 0 487466 11 0 0 25 0 5 0 507974147 237248512 29993 33554432000 4194304 4356188 140733478187568 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30504/tid=30513] ppid=30499 vsize=231688 CPUtime=4902.33 cores=4-7
/proc/30504/task/30513/stat : 30513 (plingeling) R 30499 30499 30227 0 -1 4202560 84314 1256 0 0 490219 14 0 0 25 0 5 0 507974147 237248512 29993 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=30504/tid=30514] ppid=30499 vsize=231688 CPUtime=4901.87 cores=4-7
/proc/30504/task/30514/stat : 30514 (plingeling) R 30499 30499 30227 0 -1 4202560 117377 1256 0 0 490169 18 0 0 25 0 5 0 507974147 237248512 29993 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331337 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39190.9
Current children cumulated vsize (KiB) 2125560

[startup+4962.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/180 30658
/proc/meminfo: memFree=30127020/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
[pid=30500] ppid=30499 vsize=525400 CPUtime=4962.42 cores=0
/proc/30500/stat : 30500 (cryptominisat) R 30499 30499 30227 0 -1 4202496 1143330 0 0 0 495761 481 0 0 25 0 1 0 507974140 538009600 24858 33554432000 4194304 4587354 140734713304736 18446744073709551615 4231407 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30500/statm: 131350 24858 337 96 0 127912 0
[pid=30501] ppid=30499 vsize=193760 CPUtime=4962.42 cores=1
/proc/30501/stat : 30501 (clasp) R 30499 30499 30227 0 -1 4202496 42929 0 0 0 496233 9 0 0 25 0 1 0 507974140 198410240 41587 33554432000 4194304 4785740 140735892711152 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30501/statm: 48440 41587 341 145 0 45485 0
[pid=30502] ppid=30499 vsize=460708 CPUtime=4962.25 cores=2
/proc/30502/stat : 30502 (TNM) R 30499 30499 30227 0 -1 4202496 1815 0 0 0 496223 2 0 0 25 0 1 0 507974140 471764992 1712 33554432000 134512640 134534564 4292144256 18446744073709551615 134528877 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30502/statm: 115177 1712 95 6 0 113843 0
[pid=30503] ppid=30499 vsize=696220 CPUtime=4962.42 cores=3
/proc/30503/stat : 30503 (march_hi) R 30499 30499 30227 0 -1 4202496 123421 0 0 0 496157 85 0 0 25 0 1 0 507974140 712929280 117901 33554432000 4194304 4331988 140734841743984 18446744073709551615 4210418 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30503/statm: 174055 117901 175 34 0 172497 0
[pid=30504] ppid=30499 vsize=221192 CPUtime=19821.1 cores=4-7
/proc/30504/stat : 30504 (plingeling) S 30499 30499 30227 0 -1 4202496 398283 1256 0 0 1982046 63 0 0 18 0 5 0 507974140 226500608 27369 33554432000 4194304 4356188 140733478187568 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/30504/statm: 55298 27369 161 40 0 53842 0
[pid=30504/tid=30511] ppid=30499 vsize=221192 CPUtime=4962.35 cores=4-7
/proc/30504/task/30511/stat : 30511 (plingeling) R 30499 30499 30227 0 -1 4202560 109516 1256 0 0 496218 17 0 0 25 0 5 0 507974147 226500608 27369 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30504/tid=30512] ppid=30499 vsize=221192 CPUtime=4934.44 cores=4-7
/proc/30504/task/30512/stat : 30512 (plingeling) R 30499 30499 30227 0 -1 4202560 82452 1256 0 0 493433 11 0 0 25 0 5 0 507974147 226500608 27369 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30504/tid=30513] ppid=30499 vsize=221192 CPUtime=4962.33 cores=4-7
/proc/30504/task/30513/stat : 30513 (plingeling) R 30499 30499 30227 0 -1 4202560 84314 1256 0 0 496219 14 0 0 25 0 5 0 507974147 226500608 27369 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331268 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=30504/tid=30514] ppid=30499 vsize=221192 CPUtime=4961.87 cores=4-7
/proc/30504/task/30514/stat : 30514 (plingeling) R 30499 30499 30227 0 -1 4202560 117377 1256 0 0 496169 18 0 0 25 0 5 0 507974147 226500608 27369 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331072 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39670.6
Current children cumulated vsize (KiB) 2115604



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.00 8.01 8.00 9/180 30659
/proc/meminfo: memFree=30153132/32951124 swapFree=67106388/67111528
[pid=30499] ppid=30497 vsize=18324 CPUtime=0 cores=0-7
/proc/30499/stat : 30499 (ppfolio) S 30497 30499 30227 0 -1 4202496 518 0 0 0 0 0 0 0 15 0 1 0 507974139 18763776 389 33554432000 4194304 4287625 140734818146928 18446744073709551615 212039675407 0 0 4096 65536 4259569760389300223 0 0 17 6 0 0 0
/proc/30499/statm: 4581 389 341 23 0 91 0
[pid=30500] ppid=30499 vsize=559240 CPUtime=5000.12 cores=0
/proc/30500/stat : 30500 (cryptominisat) R 30499 30499 30227 0 -1 4202496 1153133 0 0 0 499527 485 0 0 25 0 1 0 507974140 572661760 19322 33554432000 4194304 4587354 140734713304736 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30500/statm: 139810 19322 337 96 0 136372 0
[pid=30501] ppid=30499 vsize=193760 CPUtime=5000.12 cores=1
/proc/30501/stat : 30501 (clasp) R 30499 30499 30227 0 -1 4202496 42930 0 0 0 500003 9 0 0 25 0 1 0 507974140 198410240 41588 33554432000 4194304 4785740 140735892711152 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30501/statm: 48440 41588 341 145 0 45485 0
[pid=30502] ppid=30499 vsize=460708 CPUtime=4999.96 cores=2
/proc/30502/stat : 30502 (TNM) R 30499 30499 30227 0 -1 4202496 1815 0 0 0 499994 2 0 0 25 0 1 0 507974140 471764992 1712 33554432000 134512640 134534564 4292144256 18446744073709551615 134528476 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/30502/statm: 115177 1712 95 6 0 113843 0
[pid=30503] ppid=30499 vsize=696220 CPUtime=5000.12 cores=3
/proc/30503/stat : 30503 (march_hi) R 30499 30499 30227 0 -1 4202496 124343 0 0 0 499927 85 0 0 25 0 1 0 507974140 712929280 118823 33554432000 4194304 4331988 140734841743984 18446744073709551615 4287504 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30503/statm: 174055 118823 175 34 0 172497 0
[pid=30504] ppid=30499 vsize=214536 CPUtime=19971.7 cores=4-7
/proc/30504/stat : 30504 (plingeling) S 30499 30499 30227 0 -1 4202496 398667 1256 0 0 1997107 64 0 0 18 0 5 0 507974140 219684864 25705 33554432000 4194304 4356188 140733478187568 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/30504/statm: 53634 25705 161 40 0 52178 0
[pid=30504/tid=30511] ppid=30499 vsize=214536 CPUtime=5000.05 cores=4-7
/proc/30504/task/30511/stat : 30511 (plingeling) R 30499 30499 30227 0 -1 4202560 109516 1256 0 0 499988 17 0 0 25 0 5 0 507974147 219684864 25705 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30504/tid=30512] ppid=30499 vsize=214536 CPUtime=4971.94 cores=4-7
/proc/30504/task/30512/stat : 30512 (plingeling) R 30499 30499 30227 0 -1 4202560 82644 1256 0 0 497183 11 0 0 25 0 5 0 507974147 219684864 25705 33554432000 4194304 4356188 140733478187568 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30504/tid=30513] ppid=30499 vsize=214536 CPUtime=5000.04 cores=4-7
/proc/30504/task/30513/stat : 30513 (plingeling) R 30499 30499 30227 0 -1 4202560 84314 1256 0 0 499990 14 0 0 25 0 5 0 507974147 219684864 25705 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=30504/tid=30514] ppid=30499 vsize=214536 CPUtime=4999.57 cores=4-7
/proc/30504/task/30514/stat : 30514 (plingeling) R 30499 30499 30227 0 -1 4202560 117569 1256 0 0 499939 18 0 0 25 0 5 0 507974147 219684864 25705 33554432000 4194304 4356188 140733478187568 18446744073709551615 4331854 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39972
Current children cumulated vsize (KiB) 2142788

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 30499 and gives
#  childrusage.ru_utime.tv_sec=14995
#  childrusage.ru_utime.tv_usec=265373
#  childrusage.ru_stime.tv_sec=4
#  childrusage.ru_stime.tv_usec=996240
# CPU time returned by wait4() is 15000.3
# while last known CPU time is 39972
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39972
CPU user time (s): 39965.6
CPU system time (s): 6.45
CPU usage (%): 799.438
Max. virtual memory (cumulated for all children) (KiB): 6986508

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14995.3
system time used= 4.99624
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1198412
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 610
involuntary context switches= 34367

runsolver used 8.18675 second user time and 19.0941 second system time

The end

Launcher Data 

Begin job on node146 at 2011-04-30 11:26:19
IDJOB=3220047
IDBENCH=82691
IDSOLVER=1638
FILE ID=node146/3220047-1304155579
RUNJOBID= node146-1304150354-30241
PBS_JOBID= 13170900
Free space on /tmp= 73348 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k7-r85-v400-c34000-S94779963-093-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3220047-1304155579/watcher-3220047-1304155579 -o /tmp/evaluation-result-3220047-1304155579/solver-3220047-1304155579 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3220047-1304155579.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= aef8bd4e13046b42a5bc777dc56e10de
RANDOM SEED=2145699279

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30980320 kB
Buffers:        421152 kB
Cached:        1114344 kB
SwapCached:        668 kB
Active:         132728 kB
Inactive:      1461924 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30980320 kB
SwapTotal:    67111528 kB
SwapFree:     67106388 kB
Dirty:           10876 kB
Writeback:           0 kB
AnonPages:       59636 kB
Mapped:          14220 kB
Slab:           310084 kB
PageTables:       4372 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   183340 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73340 MiB
End job on node146 at 2011-04-30 12:49:41