Trace number 3222471

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39971.4 5000.02

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v600-c2556-S1885965779-049.UNKNOWN.cnf
MD5SUM465b21028c8505bfc2ee7974d056b900
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables600
Number of clauses2556
Sum of the clauses size7668
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32556
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3222471-1304165801/watcher-3222471-1304165801 -o /tmp/evaluation-result-3222471-1304165801/solver-3222471-1304165801 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3222471-1304165801.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.07 8.06 8.00 1/170 6442
/proc/meminfo: memFree=25128780/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 17 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0

[startup+0.0571229 s]
/proc/loadavg: 8.07 8.06 8.00 1/170 6442
/proc/meminfo: memFree=25128780/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 17 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100115 s]
/proc/loadavg: 8.07 8.06 8.00 1/170 6442
/proc/meminfo: memFree=25128780/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 17 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300079 s]
/proc/loadavg: 8.07 8.06 8.00 1/170 6442
/proc/meminfo: memFree=25128780/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.701007 s]
/proc/loadavg: 8.07 8.06 8.00 1/170 6442
/proc/meminfo: memFree=25128780/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50086 s]
/proc/loadavg: 8.07 8.06 8.00 9/180 6457
/proc/meminfo: memFree=25110844/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
[pid=6443] ppid=6442 vsize=1436220 CPUtime=1.49 cores=0
/proc/6443/stat : 6443 (cryptominisat) R 6442 6442 5706 0 -1 4202496 3929 0 0 0 149 0 0 0 25 0 1 0 508995990 1470689280 2623 33554432000 4194304 4587354 140735982015808 18446744073709551615 4365232 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6443/statm: 359055 2623 337 96 0 355617 0
[pid=6444] ppid=6442 vsize=14296 CPUtime=1.49 cores=1
/proc/6444/stat : 6444 (clasp) R 6442 6442 5706 0 -1 4202496 1043 0 0 0 149 0 0 0 25 0 1 0 508995990 14639104 930 33554432000 4194304 4785740 140733744483920 18446744073709551615 4287008 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6444/statm: 3574 930 340 145 0 619 0
[pid=6445] ppid=6442 vsize=455960 CPUtime=1.49 cores=2
/proc/6445/stat : 6445 (TNM) R 6442 6442 5706 0 -1 4202496 355 0 0 0 149 0 0 0 20 0 1 0 508995990 466903040 252 33554432000 134512640 134534564 4291975536 18446744073709551615 134529031 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/6445/statm: 113990 252 95 6 0 112656 0
[pid=6446] ppid=6442 vsize=8236 CPUtime=1.49 cores=3
/proc/6446/stat : 6446 (march_hi) R 6442 6442 5706 0 -1 4202496 661 0 0 0 149 0 0 0 20 0 1 0 508995990 8433664 554 33554432000 4194304 4331988 140737312947280 18446744073709551615 4226333 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6446/statm: 2059 554 173 34 0 501 0
[pid=6447] ppid=6442 vsize=122876 CPUtime=5.92 cores=4-7
/proc/6447/stat : 6447 (plingeling) S 6442 6442 5706 0 -1 4202496 4092 1257 0 0 592 0 0 0 18 0 5 0 508995990 125825024 2841 33554432000 4194304 4356188 140736289589520 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6447/statm: 30719 2841 164 40 0 29263 0
[pid=6447/tid=6454] ppid=6442 vsize=122876 CPUtime=1.48 cores=4-7
/proc/6447/task/6454/stat : 6454 (plingeling) R 6442 6442 5706 0 -1 4202560 1014 1257 0 0 148 0 0 0 23 0 5 0 508995991 125825024 2841 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331939 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6447/tid=6455] ppid=6442 vsize=122876 CPUtime=1.46 cores=4-7
/proc/6447/task/6455/stat : 6455 (plingeling) R 6442 6442 5706 0 -1 4202560 752 1257 0 0 146 0 0 0 23 0 5 0 508995991 125825024 2841 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6447/tid=6456] ppid=6442 vsize=122876 CPUtime=1.48 cores=4-7
/proc/6447/task/6456/stat : 6456 (plingeling) R 6442 6442 5706 0 -1 4202560 822 1257 0 0 148 0 0 0 25 0 5 0 508995991 125825024 2841 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331371 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6447/tid=6457] ppid=6442 vsize=122876 CPUtime=1.47 cores=4-7
/proc/6447/task/6457/stat : 6457 (plingeling) R 6442 6442 5706 0 -1 4202560 977 1257 0 0 147 0 0 0 25 0 5 0 508995991 125825024 2841 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.88
Current children cumulated vsize (KiB) 2055912

[startup+3.10058 s]
/proc/loadavg: 8.07 8.06 8.00 9/180 6457
/proc/meminfo: memFree=25103252/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
[pid=6443] ppid=6442 vsize=1437728 CPUtime=3.09 cores=0
/proc/6443/stat : 6443 (cryptominisat) R 6442 6442 5706 0 -1 4202496 4963 0 0 0 309 0 0 0 25 0 1 0 508995990 1472233472 2759 33554432000 4194304 4587354 140735982015808 18446744073709551615 4238591 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6443/statm: 359432 2759 337 96 0 355994 0
[pid=6444] ppid=6442 vsize=15244 CPUtime=3.09 cores=1
/proc/6444/stat : 6444 (clasp) R 6442 6442 5706 0 -1 4202496 1259 0 0 0 309 0 0 0 25 0 1 0 508995990 15609856 1146 33554432000 4194304 4785740 140733744483920 18446744073709551615 4287053 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6444/statm: 3811 1146 340 145 0 856 0
[pid=6445] ppid=6442 vsize=455960 CPUtime=3.09 cores=2
/proc/6445/stat : 6445 (TNM) R 6442 6442 5706 0 -1 4202496 355 0 0 0 309 0 0 0 25 0 1 0 508995990 466903040 252 33554432000 134512640 134534564 4291975536 18446744073709551615 134528620 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/6445/statm: 113990 252 95 6 0 112656 0
[pid=6446] ppid=6442 vsize=8772 CPUtime=3.09 cores=3
/proc/6446/stat : 6446 (march_hi) R 6442 6442 5706 0 -1 4202496 738 0 0 0 309 0 0 0 25 0 1 0 508995990 8982528 631 33554432000 4194304 4331988 140737312947280 18446744073709551615 4263148 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6446/statm: 2193 631 173 34 0 635 0
[pid=6447] ppid=6442 vsize=122884 CPUtime=12.31 cores=4-7
/proc/6447/stat : 6447 (plingeling) S 6442 6442 5706 0 -1 4202496 5208 1257 0 0 1231 0 0 0 18 0 5 0 508995990 125833216 2843 33554432000 4194304 4356188 140736289589520 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6447/statm: 30721 2843 164 40 0 29265 0
[pid=6447/tid=6454] ppid=6442 vsize=122884 CPUtime=3.08 cores=4-7
/proc/6447/task/6454/stat : 6454 (plingeling) R 6442 6442 5706 0 -1 4202560 1395 1257 0 0 308 0 0 0 25 0 5 0 508995991 125833216 2843 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331086 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6447/tid=6455] ppid=6442 vsize=122884 CPUtime=3.06 cores=4-7
/proc/6447/task/6455/stat : 6455 (plingeling) R 6442 6442 5706 0 -1 4202560 816 1257 0 0 306 0 0 0 25 0 5 0 508995991 125833216 2843 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6447/tid=6456] ppid=6442 vsize=122884 CPUtime=3.08 cores=4-7
/proc/6447/task/6456/stat : 6456 (plingeling) R 6442 6442 5706 0 -1 4202560 1143 1257 0 0 308 0 0 0 25 0 5 0 508995991 125833216 2843 33554432000 4194304 4356188 140736289589520 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6447/tid=6457] ppid=6442 vsize=122884 CPUtime=3.07 cores=4-7
/proc/6447/task/6457/stat : 6457 (plingeling) R 6442 6442 5706 0 -1 4202560 1327 1257 0 0 307 0 0 0 25 0 5 0 508995991 125833216 2843 33554432000 4194304 4356188 140736289589520 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.67
Current children cumulated vsize (KiB) 2058912

[startup+6.301 s]
/proc/loadavg: 8.07 8.06 8.00 9/180 6457
/proc/meminfo: memFree=25094312/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
[pid=6443] ppid=6442 vsize=1436212 CPUtime=6.29 cores=0
/proc/6443/stat : 6443 (cryptominisat) R 6442 6442 5706 0 -1 4202496 7330 0 0 0 628 1 0 0 25 0 1 0 508995990 1470681088 2827 33554432000 4194304 4587354 140735982015808 18446744073709551615 4238755 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6443/statm: 359053 2827 337 96 0 355615 0
[pid=6444] ppid=6442 vsize=16312 CPUtime=6.29 cores=1
/proc/6444/stat : 6444 (clasp) R 6442 6442 5706 0 -1 4202496 1542 0 0 0 629 0 0 0 25 0 1 0 508995990 16703488 1429 33554432000 4194304 4785740 140733744483920 18446744073709551615 4287053 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6444/statm: 4078 1429 340 145 0 1123 0
[pid=6445] ppid=6442 vsize=455960 CPUtime=6.29 cores=2
/proc/6445/stat : 6445 (TNM) R 6442 6442 5706 0 -1 4202496 355 0 0 0 629 0 0 0 25 0 1 0 508995990 466903040 252 33554432000 134512640 134534564 4291975536 18446744073709551615 134530770 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/6445/statm: 113990 252 95 6 0 112656 0
[pid=6446] ppid=6442 vsize=9112 CPUtime=6.29 cores=3
/proc/6446/stat : 6446 (march_hi) R 6442 6442 5706 0 -1 4202496 874 0 0 0 629 0 0 0 25 0 1 0 508995990 9330688 767 33554432000 4194304 4331988 140737312947280 18446744073709551615 4229684 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6446/statm: 2278 767 173 34 0 720 0
[pid=6447] ppid=6442 vsize=124956 CPUtime=25.09 cores=4-7
/proc/6447/stat : 6447 (plingeling) S 6442 6442 5706 0 -1 4202496 7902 1257 0 0 2509 0 0 0 18 0 5 0 508995990 127954944 3361 33554432000 4194304 4356188 140736289589520 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6447/statm: 31239 3361 164 40 0 29783 0
[pid=6447/tid=6454] ppid=6442 vsize=124956 CPUtime=6.28 cores=4-7
/proc/6447/task/6454/stat : 6454 (plingeling) R 6442 6442 5706 0 -1 4202560 2004 1257 0 0 628 0 0 0 25 0 5 0 508995991 127954944 3361 33554432000 4194304 4356188 140736289589520 18446744073709551615 4210817 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6447/tid=6455] ppid=6442 vsize=124956 CPUtime=6.26 cores=4-7
/proc/6447/task/6455/stat : 6455 (plingeling) R 6442 6442 5706 0 -1 4202560 1522 1257 0 0 626 0 0 0 25 0 5 0 508995991 127954944 3361 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331165 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6447/tid=6456] ppid=6442 vsize=124956 CPUtime=6.28 cores=4-7
/proc/6447/task/6456/stat : 6456 (plingeling) R 6442 6442 5706 0 -1 4202560 1848 1257 0 0 628 0 0 0 25 0 5 0 508995991 127954944 3361 33554432000 4194304 4356188 140736289589520 18446744073709551615 4234227 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6447/tid=6457] ppid=6442 vsize=124956 CPUtime=6.25 cores=4-7
/proc/6447/task/6457/stat : 6457 (plingeling) R 6442 6442 5706 0 -1 4202560 2001 1257 0 0 625 0 0 0 25 0 5 0 508995991 127954944 3361 33554432000 4194304 4356188 140736289589520 18446744073709551615 4280698 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.25
Current children cumulated vsize (KiB) 2060876

[startup+12.7009 s]
/proc/loadavg: 8.06 8.06 8.00 9/180 6457
/proc/meminfo: memFree=25083532/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
[pid=6443] ppid=6442 vsize=1444204 CPUtime=12.69 cores=0
/proc/6443/stat : 6443 (cryptominisat) R 6442 6442 5706 0 -1 4202496 12026 0 0 0 1267 2 0 0 25 0 1 0 508995990 1478864896 4734 33554432000 4194304 4587354 140735982015808 18446744073709551615 4238418 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6443/statm: 361051 4734 337 96 0 357613 0
[pid=6444] ppid=6442 vsize=17736 CPUtime=12.69 cores=1
/proc/6444/stat : 6444 (clasp) R 6442 6442 5706 0 -1 4202496 1896 0 0 0 1269 0 0 0 25 0 1 0 508995990 18161664 1783 33554432000 4194304 4785740 140733744483920 18446744073709551615 4553088 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6444/statm: 4434 1783 341 145 0 1479 0
[pid=6445] ppid=6442 vsize=455960 CPUtime=12.69 cores=2
/proc/6445/stat : 6445 (TNM) R 6442 6442 5706 0 -1 4202496 355 0 0 0 1269 0 0 0 25 0 1 0 508995990 466903040 252 33554432000 134512640 134534564 4291975536 18446744073709551615 134529031 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/6445/statm: 113990 252 95 6 0 112656 0
[pid=6446] ppid=6442 vsize=10936 CPUtime=12.69 cores=3
/proc/6446/stat : 6446 (march_hi) R 6442 6442 5706 0 -1 4202496 1150 0 0 0 1269 0 0 0 25 0 1 0 508995990 11198464 1043 33554432000 4194304 4331988 140737312947280 18446744073709551615 4229623 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6446/statm: 2734 1043 173 34 0 1176 0
[pid=6447] ppid=6442 vsize=130468 CPUtime=50.65 cores=4-7
/proc/6447/stat : 6447 (plingeling) S 6442 6442 5706 0 -1 4202496 12734 1257 0 0 5064 1 0 0 18 0 5 0 508995990 133599232 4737 33554432000 4194304 4356188 140736289589520 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6447/statm: 32617 4737 164 40 0 31161 0
[pid=6447/tid=6454] ppid=6442 vsize=130468 CPUtime=12.68 cores=4-7
/proc/6447/task/6454/stat : 6454 (plingeling) R 6442 6442 5706 0 -1 4202560 3125 1257 0 0 1268 0 0 0 25 0 5 0 508995991 133599232 4737 33554432000 4194304 4356188 140736289589520 18446744073709551615 4332225 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6447/tid=6455] ppid=6442 vsize=130468 CPUtime=12.66 cores=4-7
/proc/6447/task/6455/stat : 6455 (plingeling) R 6442 6442 5706 0 -1 4202560 2739 1257 0 0 1266 0 0 0 25 0 5 0 508995991 133599232 4737 33554432000 4194304 4356188 140736289589520 18446744073709551615 4332327 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6447/tid=6456] ppid=6442 vsize=130468 CPUtime=12.68 cores=4-7
/proc/6447/task/6456/stat : 6456 (plingeling) R 6442 6442 5706 0 -1 4202560 3253 1257 0 0 1268 0 0 0 25 0 5 0 508995991 133599232 4737 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331151 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6447/tid=6457] ppid=6442 vsize=130468 CPUtime=12.61 cores=4-7
/proc/6447/task/6457/stat : 6457 (plingeling) R 6442 6442 5706 0 -1 4202560 3090 1257 0 0 1261 0 0 0 25 0 5 0 508995991 133599232 4737 33554432000 4194304 4356188 140736289589520 18446744073709551615 4332003 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.41
Current children cumulated vsize (KiB) 2077628

[startup+25.5006 s]
/proc/loadavg: 8.05 8.05 8.00 9/180 6458
/proc/meminfo: memFree=25068664/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
[pid=6443] ppid=6442 vsize=1445264 CPUtime=25.49 cores=0
/proc/6443/stat : 6443 (cryptominisat) R 6442 6442 5706 0 -1 4202496 16834 0 0 0 2546 3 0 0 25 0 1 0 508995990 1479950336 5176 33554432000 4194304 4587354 140735982015808 18446744073709551615 4238334 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=6445] ppid=6442 vsize=455964 CPUtime=4781.45 cores=2
/proc/6445/stat : 6445 (TNM) R 6442 6442 5706 0 -1 4202496 364 0 0 0 478144 1 0 0 25 0 1 0 508995990 466907136 261 33554432000 134512640 134534564 4291975536 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/6445/statm: 113991 261 103 6 0 112657 0
[pid=6446] ppid=6442 vsize=596440 CPUtime=4782.41 cores=3
/proc/6446/stat : 6446 (march_hi) R 6442 6442 5706 0 -1 4202496 143658 0 0 0 478127 114 0 0 25 0 1 0 508995990 610754560 143551 33554432000 4194304 4331988 140737312947280 18446744073709551615 4226450 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6446/statm: 149110 143551 173 34 0 147552 0
[pid=6447] ppid=6442 vsize=275556 CPUtime=19102.4 cores=4-7
/proc/6447/stat : 6447 (plingeling) S 6442 6442 5706 0 -1 4202496 933168 1257 0 0 1910099 144 0 0 18 0 5 0 508995990 282169344 41011 33554432000 4194304 4356188 140736289589520 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6447/statm: 68889 41011 164 40 0 67433 0
[pid=6447/tid=6454] ppid=6442 vsize=275556 CPUtime=4781.75 cores=4-7
/proc/6447/task/6454/stat : 6454 (plingeling) R 6442 6442 5706 0 -1 4202560 220728 1257 0 0 478142 33 0 0 25 0 5 0 508995991 282169344 41011 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6447/tid=6455] ppid=6442 vsize=275556 CPUtime=4781.91 cores=4-7
/proc/6447/task/6455/stat : 6455 (plingeling) R 6442 6442 5706 0 -1 4202560 232504 1257 0 0 478157 34 0 0 25 0 5 0 508995991 282169344 41011 33554432000 4194304 4356188 140736289589520 18446744073709551615 4332165 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6447/tid=6456] ppid=6442 vsize=275556 CPUtime=4782.41 cores=4-7
/proc/6447/task/6456/stat : 6456 (plingeling) R 6442 6442 5706 0 -1 4202560 239097 1257 0 0 478202 39 0 0 25 0 5 0 508995991 282169344 41011 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6447/tid=6457] ppid=6442 vsize=275556 CPUtime=4756.32 cores=4-7
/proc/6447/task/6457/stat : 6457 (plingeling) R 6442 6442 5706 0 -1 4202560 240312 1257 0 0 475596 36 0 0 25 0 5 0 508995991 282169344 41011 33554432000 4194304 4356188 140736289589520 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38230.9
Current children cumulated vsize (KiB) 2945100

[startup+4842.3 s]
/proc/loadavg: 8.07 8.06 8.00 9/180 6601
/proc/meminfo: memFree=24203772/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
[pid=6443] ppid=6442 vsize=1536696 CPUtime=4842.4 cores=0
/proc/6443/stat : 6443 (cryptominisat) R 6442 6442 5706 0 -1 4202496 272525 0 0 0 484119 121 0 0 25 0 1 0 508995990 1573576704 31662 33554432000 4194304 4587354 140735982015808 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6443/statm: 384174 31662 337 96 0 380736 0
[pid=6444] ppid=6442 vsize=62120 CPUtime=4842.27 cores=1
/proc/6444/stat : 6444 (clasp) R 6442 6442 5706 0 -1 4202496 12078 0 0 0 484224 3 0 0 25 0 1 0 508995990 63610880 11965 33554432000 4194304 4785740 140733744483920 18446744073709551615 4287158 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6444/statm: 15530 11965 341 145 0 12575 0
[pid=6445] ppid=6442 vsize=455964 CPUtime=4841.46 cores=2
/proc/6445/stat : 6445 (TNM) R 6442 6442 5706 0 -1 4202496 364 0 0 0 484145 1 0 0 25 0 1 0 508995990 466907136 261 33554432000 134512640 134534564 4291975536 18446744073709551615 134528451 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/6445/statm: 113991 261 103 6 0 112657 0
[pid=6446] ppid=6442 vsize=820204 CPUtime=4842.42 cores=3
/proc/6446/stat : 6446 (march_hi) R 6442 6442 5706 0 -1 4202496 145542 0 0 0 484126 116 0 0 25 0 1 0 508995990 839888896 145435 33554432000 4194304 4331988 140737312947280 18446744073709551615 4286225 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6446/statm: 205051 145435 173 34 0 203493 0
[pid=6447] ppid=6442 vsize=280036 CPUtime=19342.1 cores=4-7
/proc/6447/stat : 6447 (plingeling) S 6442 6442 5706 0 -1 4202496 944048 1257 0 0 1934064 146 0 0 18 0 5 0 508995990 286756864 42131 33554432000 4194304 4356188 140736289589520 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6447/statm: 70009 42131 164 40 0 68553 0
[pid=6447/tid=6454] ppid=6442 vsize=280036 CPUtime=4841.75 cores=4-7
/proc/6447/task/6454/stat : 6454 (plingeling) R 6442 6442 5706 0 -1 4202560 222552 1257 0 0 484142 33 0 0 25 0 5 0 508995991 286756864 42131 33554432000 4194304 4356188 140736289589520 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6447/tid=6455] ppid=6442 vsize=280036 CPUtime=4841.89 cores=4-7
/proc/6447/task/6455/stat : 6455 (plingeling) R 6442 6442 5706 0 -1 4202560 237912 1257 0 0 484155 34 0 0 25 0 5 0 508995991 286756864 42131 33554432000 4194304 4356188 140736289589520 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6447/tid=6456] ppid=6442 vsize=280036 CPUtime=4842.41 cores=4-7
/proc/6447/task/6456/stat : 6456 (plingeling) R 6442 6442 5706 0 -1 4202560 241305 1257 0 0 484202 39 0 0 25 0 5 0 508995991 286756864 42131 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6447/tid=6457] ppid=6442 vsize=280036 CPUtime=4816 cores=4-7
/proc/6447/task/6457/stat : 6457 (plingeling) R 6442 6442 5706 0 -1 4202560 241752 1257 0 0 481563 37 0 0 25 0 5 0 508995991 286756864 42131 33554432000 4194304 4356188 140736289589520 18446744073709551615 4279490 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38710.7
Current children cumulated vsize (KiB) 3173344

[startup+4902.3 s]
/proc/loadavg: 8.06 8.06 8.00 9/180 6603
/proc/meminfo: memFree=24183264/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
[pid=6443] ppid=6442 vsize=1559368 CPUtime=4902.39 cores=0
/proc/6443/stat : 6443 (cryptominisat) R 6442 6442 5706 0 -1 4202496 277965 0 0 0 490117 122 0 0 25 0 1 0 508995990 1596792832 35318 33554432000 4194304 4587354 140735982015808 18446744073709551615 4238552 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6443/statm: 389842 35318 337 96 0 386404 0
[pid=6444] ppid=6442 vsize=62316 CPUtime=4902.27 cores=1
/proc/6444/stat : 6444 (clasp) R 6442 6442 5706 0 -1 4202496 12106 0 0 0 490224 3 0 0 25 0 1 0 508995990 63811584 11993 33554432000 4194304 4785740 140733744483920 18446744073709551615 4553216 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6444/statm: 15579 11993 341 145 0 12624 0
[pid=6445] ppid=6442 vsize=455964 CPUtime=4901.46 cores=2
/proc/6445/stat : 6445 (TNM) R 6442 6442 5706 0 -1 4202496 364 0 0 0 490145 1 0 0 25 0 1 0 508995990 466907136 261 33554432000 134512640 134534564 4291975536 18446744073709551615 134529874 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/6445/statm: 113991 261 103 6 0 112657 0
[pid=6446] ppid=6442 vsize=820204 CPUtime=4902.41 cores=3
/proc/6446/stat : 6446 (march_hi) R 6442 6442 5706 0 -1 4202496 147309 0 0 0 490124 117 0 0 25 0 1 0 508995990 839888896 147202 33554432000 4194304 4331988 140737312947280 18446744073709551615 4229294 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6446/statm: 205051 147202 173 34 0 203493 0
[pid=6447] ppid=6442 vsize=281320 CPUtime=19581.8 cores=4-7
/proc/6447/stat : 6447 (plingeling) S 6442 6442 5706 0 -1 4202496 948465 1257 0 0 1958031 146 0 0 18 0 5 0 508995990 288071680 42452 33554432000 4194304 4356188 140736289589520 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6447/statm: 70330 42452 164 40 0 68874 0
[pid=6447/tid=6454] ppid=6442 vsize=281320 CPUtime=4901.76 cores=4-7
/proc/6447/task/6454/stat : 6454 (plingeling) R 6442 6442 5706 0 -1 4202560 223865 1257 0 0 490142 34 0 0 25 0 5 0 508995991 288071680 42452 33554432000 4194304 4356188 140736289589520 18446744073709551615 4332239 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6447/tid=6455] ppid=6442 vsize=281320 CPUtime=4901.9 cores=4-7
/proc/6447/task/6455/stat : 6455 (plingeling) R 6442 6442 5706 0 -1 4202560 239064 1257 0 0 490155 35 0 0 25 0 5 0 508995991 288071680 42452 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6447/tid=6456] ppid=6442 vsize=281320 CPUtime=4902.41 cores=4-7
/proc/6447/task/6456/stat : 6456 (plingeling) R 6442 6442 5706 0 -1 4202560 242489 1257 0 0 490202 39 0 0 25 0 5 0 508995991 288071680 42452 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6447/tid=6457] ppid=6442 vsize=281320 CPUtime=4875.68 cores=4-7
/proc/6447/task/6457/stat : 6457 (plingeling) R 6442 6442 5706 0 -1 4202560 242520 1257 0 0 487531 37 0 0 25 0 5 0 508995991 288071680 42452 33554432000 4194304 4356188 140736289589520 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39190.3
Current children cumulated vsize (KiB) 3197496

[startup+4962.3 s]
/proc/loadavg: 8.06 8.06 8.00 9/180 6604
/proc/meminfo: memFree=24208884/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
[pid=6443] ppid=6442 vsize=1536696 CPUtime=4962.4 cores=0
/proc/6443/stat : 6443 (cryptominisat) R 6442 6442 5706 0 -1 4202496 279227 0 0 0 496117 123 0 0 25 0 1 0 508995990 1573576704 30046 33554432000 4194304 4587354 140735982015808 18446744073709551615 4237681 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6443/statm: 384174 30046 337 96 0 380736 0
[pid=6444] ppid=6442 vsize=62316 CPUtime=4962.27 cores=1
/proc/6444/stat : 6444 (clasp) R 6442 6442 5706 0 -1 4202496 12123 0 0 0 496224 3 0 0 25 0 1 0 508995990 63811584 12010 33554432000 4194304 4785740 140733744483920 18446744073709551615 4286976 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6444/statm: 15579 12010 341 145 0 12624 0
[pid=6445] ppid=6442 vsize=455964 CPUtime=4961.46 cores=2
/proc/6445/stat : 6445 (TNM) R 6442 6442 5706 0 -1 4202496 364 0 0 0 496145 1 0 0 25 0 1 0 508995990 466907136 261 33554432000 134512640 134534564 4291975536 18446744073709551615 134530332 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/6445/statm: 113991 261 103 6 0 112657 0
[pid=6446] ppid=6442 vsize=820204 CPUtime=4962.42 cores=3
/proc/6446/stat : 6446 (march_hi) R 6442 6442 5706 0 -1 4202496 149140 0 0 0 496123 119 0 0 25 0 1 0 508995990 839888896 149033 33554432000 4194304 4331988 140737312947280 18446744073709551615 4211435 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6446/statm: 205051 149033 173 34 0 203493 0
[pid=6447] ppid=6442 vsize=266220 CPUtime=19821.4 cores=4-7
/proc/6447/stat : 6447 (plingeling) S 6442 6442 5706 0 -1 4202496 958706 1257 0 0 1981997 148 0 0 18 0 5 0 508995990 272609280 38677 33554432000 4194304 4356188 140736289589520 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6447/statm: 66555 38677 164 40 0 65099 0
[pid=6447/tid=6454] ppid=6442 vsize=266220 CPUtime=4961.76 cores=4-7
/proc/6447/task/6454/stat : 6454 (plingeling) R 6442 6442 5706 0 -1 4202560 225177 1257 0 0 496142 34 0 0 25 0 5 0 508995991 272609280 38677 33554432000 4194304 4356188 140736289589520 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6447/tid=6455] ppid=6442 vsize=266220 CPUtime=4961.89 cores=4-7
/proc/6447/task/6455/stat : 6455 (plingeling) R 6442 6442 5706 0 -1 4202560 240376 1257 0 0 496154 35 0 0 25 0 5 0 508995991 272609280 38677 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6447/tid=6456] ppid=6442 vsize=266220 CPUtime=4962.42 cores=4-7
/proc/6447/task/6456/stat : 6456 (plingeling) R 6442 6442 5706 0 -1 4202560 244186 1257 0 0 496202 40 0 0 25 0 5 0 508995991 272609280 38677 33554432000 4194304 4356188 140736289589520 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6447/tid=6457] ppid=6442 vsize=266220 CPUtime=4935.34 cores=4-7
/proc/6447/task/6457/stat : 6457 (plingeling) R 6442 6442 5706 0 -1 4202560 248440 1257 0 0 493497 37 0 0 25 0 5 0 508995991 272609280 38677 33554432000 4194304 4356188 140736289589520 18446744073709551615 4279717 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39670
Current children cumulated vsize (KiB) 3159724



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.08 8.07 8.00 9/180 6605
/proc/meminfo: memFree=24182692/32951124 swapFree=67111524/67111528
[pid=6442] ppid=6440 vsize=18324 CPUtime=0 cores=0-7
/proc/6442/stat : 6442 (ppfolio) S 6440 6442 5706 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508995989 18763776 389 33554432000 4194304 4287625 140735217291584 18446744073709551615 233510317583 0 0 4096 65536 11001907415623925759 0 0 17 2 0 0 0
/proc/6442/statm: 4581 389 341 23 0 91 0
[pid=6443] ppid=6442 vsize=1536696 CPUtime=5000.1 cores=0
/proc/6443/stat : 6443 (cryptominisat) R 6442 6442 5706 0 -1 4202496 280533 0 0 0 499886 124 0 0 25 0 1 0 508995990 1573576704 31314 33554432000 4194304 4587354 140735982015808 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/6443/statm: 384174 31314 337 96 0 380736 0
[pid=6444] ppid=6442 vsize=62484 CPUtime=4999.97 cores=1
/proc/6444/stat : 6444 (clasp) R 6442 6442 5706 0 -1 4202496 12143 0 0 0 499994 3 0 0 25 0 1 0 508995990 63983616 12030 33554432000 4194304 4785740 140733744483920 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/6444/statm: 15621 12030 341 145 0 12666 0
[pid=6445] ppid=6442 vsize=455964 CPUtime=4999.17 cores=2
/proc/6445/stat : 6445 (TNM) R 6442 6442 5706 0 -1 4202496 364 0 0 0 499916 1 0 0 25 0 1 0 508995990 466907136 261 33554432000 134512640 134534564 4291975536 18446744073709551615 134528913 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/6445/statm: 113991 261 103 6 0 112657 0
[pid=6446] ppid=6442 vsize=820204 CPUtime=5000.13 cores=3
/proc/6446/stat : 6446 (march_hi) R 6442 6442 5706 0 -1 4202496 150254 0 0 0 499893 120 0 0 25 0 1 0 508995990 839888896 150147 33554432000 4194304 4331988 140737312947280 18446744073709551615 4229471 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/6446/statm: 205051 150147 173 34 0 203493 0
[pid=6447] ppid=6442 vsize=285164 CPUtime=19972.1 cores=4-7
/proc/6447/stat : 6447 (plingeling) S 6442 6442 5706 0 -1 4202496 963442 1257 0 0 1997058 148 0 0 18 0 5 0 508995990 292007936 43413 33554432000 4194304 4356188 140736289589520 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/6447/statm: 71291 43413 164 40 0 69835 0
[pid=6447/tid=6454] ppid=6442 vsize=285164 CPUtime=4999.46 cores=4-7
/proc/6447/task/6454/stat : 6454 (plingeling) R 6442 6442 5706 0 -1 4202560 229561 1257 0 0 499912 34 0 0 25 0 5 0 508995991 292007936 43413 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6447/tid=6455] ppid=6442 vsize=285164 CPUtime=4999.59 cores=4-7
/proc/6447/task/6455/stat : 6455 (plingeling) R 6442 6442 5706 0 -1 4202560 240632 1257 0 0 499924 35 0 0 25 0 5 0 508995991 292007936 43413 33554432000 4194304 4356188 140736289589520 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=6447/tid=6456] ppid=6442 vsize=285164 CPUtime=5000.12 cores=4-7
/proc/6447/task/6456/stat : 6456 (plingeling) R 6442 6442 5706 0 -1 4202560 244218 1257 0 0 499972 40 0 0 25 0 5 0 508995991 292007936 43413 33554432000 4194304 4356188 140736289589520 18446744073709551615 4331589 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=6447/tid=6457] ppid=6442 vsize=285164 CPUtime=4972.84 cores=4-7
/proc/6447/task/6457/stat : 6457 (plingeling) R 6442 6442 5706 0 -1 4202560 248504 1257 0 0 497247 37 0 0 25 0 5 0 508995991 292007936 43413 33554432000 4194304 4356188 140736289589520 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39971.4
Current children cumulated vsize (KiB) 3178836

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 6442 and gives
#  childrusage.ru_utime.tv_sec=14997
#  childrusage.ru_utime.tv_usec=980960
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=309800
# CPU time returned by wait4() is 14999.3
# while last known CPU time is 39971.4
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39971.4
CPU user time (s): 39967.5
CPU system time (s): 3.96
CPU usage (%): 799.426
Max. virtual memory (cumulated for all children) (KiB): 4669740

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14998
system time used= 1.3098
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 293575
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 908
involuntary context switches= 90946

runsolver used 8.00178 second user time and 19.2761 second system time

The end

Launcher Data 

Begin job on node145 at 2011-04-30 14:16:41
IDJOB=3222471
IDBENCH=83095
IDSOLVER=1638
FILE ID=node145/3222471-1304165801
RUNJOBID= node145-1304154457-5720
PBS_JOBID= 13170907
Free space on /tmp= 73028 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v600-c2556-S1885965779-049.UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3222471-1304165801/watcher-3222471-1304165801 -o /tmp/evaluation-result-3222471-1304165801/solver-3222471-1304165801 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3222471-1304165801.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 465b21028c8505bfc2ee7974d056b900
RANDOM SEED=740397732

node145.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25128936 kB
Buffers:       1291792 kB
Cached:        5935612 kB
SwapCached:          4 kB
Active:         860404 kB
Inactive:      6430708 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25128936 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           10092 kB
Writeback:           0 kB
AnonPages:       63296 kB
Mapped:          14728 kB
Slab:           466512 kB
PageTables:       4252 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   182188 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73020 MiB
End job on node145 at 2011-04-30 15:40:03