Trace number 3219729

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 34992.1 5000.02

General information on the benchmark

Namerandom/large/
unif-k7-r85-v200-c17000-S970203146-047-UNKNOWN.cnf
MD5SUM635ea520fe16a78d41337039f98997e5
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark580.671
Satisfiable
(Un)Satisfiability was proved
Number of variables200
Number of clauses17000
Sum of the clauses size119000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 517000

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3219729-1304152794/watcher-3219729-1304152794 -o /tmp/evaluation-result-3219729-1304152794/solver-3219729-1304152794 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3219729-1304152794.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.11 8.05 8.00 2/170 10040
/proc/meminfo: memFree=28992304/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=0 cores=0-7
/proc/10040/stat : 10040 (ppfolio) R 10038 10040 9630 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 526877168 18763776 389 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/10040/statm: 4581 389 341 23 0 91 0

[startup+0.0753881 s]
/proc/loadavg: 8.11 8.05 8.00 2/170 10040
/proc/meminfo: memFree=28992304/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=0 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 526877168 18763776 389 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 4588166369492598783 0 0 17 6 0 0 0
/proc/10040/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100381 s]
/proc/loadavg: 8.11 8.05 8.00 2/170 10040
/proc/meminfo: memFree=28992304/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=0 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 526877168 18763776 389 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 4588166369492598783 0 0 17 6 0 0 0
/proc/10040/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300347 s]
/proc/loadavg: 8.11 8.05 8.00 2/170 10040
/proc/meminfo: memFree=28992304/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=0 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 526877168 18763776 389 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 4588166369492598783 0 0 17 6 0 0 0
/proc/10040/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700273 s]
/proc/loadavg: 8.11 8.05 8.00 2/170 10040
/proc/meminfo: memFree=28992304/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=0 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 521 0 0 0 0 0 0 0 19 0 1 0 526877168 18763776 389 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 4588166369492598783 0 0 17 6 0 0 0
/proc/10040/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50113 s]
/proc/loadavg: 8.11 8.05 8.00 9/180 10055
/proc/meminfo: memFree=28967300/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=0 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526877168 18763776 389 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 4588166369492598783 0 0 17 6 0 0 0
/proc/10040/statm: 4581 389 341 23 0 91 0
[pid=10041] ppid=10040 vsize=845140 CPUtime=1.49 cores=0
/proc/10041/stat : 10041 (cryptominisat) R 10040 10040 9630 0 -1 4202496 5046 0 0 0 149 0 0 0 20 0 1 0 526877168 865423360 1554 33554432000 4194304 4587354 140733490335712 18446744073709551615 4238947 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10041/statm: 211285 1554 327 96 0 207847 0
[pid=10042] ppid=10040 vsize=18520 CPUtime=1.49 cores=1
/proc/10042/stat : 10042 (clasp) R 10040 10040 9630 0 -1 4202496 2110 0 0 0 149 0 0 0 20 0 1 0 526877168 18964480 1967 33554432000 4194304 4785740 140737258779488 18446744073709551615 4287104 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10042/statm: 4630 1967 341 145 0 1675 0
[pid=10043] ppid=10040 vsize=453976 CPUtime=1.49 cores=2
/proc/10043/stat : 10043 (TNM) R 10040 10040 9630 0 -1 4202496 843 0 0 0 149 0 0 0 20 0 1 0 526877168 464871424 740 33554432000 134512640 134534564 4293247376 18446744073709551615 134524561 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10043/statm: 113494 740 91 6 0 112160 0
[pid=10044] ppid=10040 vsize=11768 CPUtime=1.49 cores=3
/proc/10044/stat : 10044 (march_hi) R 10040 10040 9630 0 -1 4202496 3827 0 0 0 148 1 0 0 20 0 1 0 526877168 12050432 1222 33554432000 4194304 4331988 140736650348672 18446744073709551615 4210416 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10044/statm: 2942 1222 174 34 0 1384 0
[pid=10045] ppid=10040 vsize=124632 CPUtime=5.83 cores=4-7
/proc/10045/stat : 10045 (plingeling) S 10040 10040 9630 0 -1 4202496 5035 1252 0 0 582 1 0 0 25 0 5 0 526877168 127623168 3233 33554432000 4194304 4356188 140734915558016 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10045/statm: 31158 3233 162 40 0 29702 0
[pid=10045/tid=10052] ppid=10040 vsize=124632 CPUtime=1.46 cores=4-7
/proc/10045/task/10052/stat : 10052 (plingeling) R 10040 10040 9630 0 -1 4202560 616 1252 0 0 146 0 0 0 25 0 5 0 526877172 127623168 3233 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10045/tid=10053] ppid=10040 vsize=124632 CPUtime=1.45 cores=4-7
/proc/10045/task/10053/stat : 10053 (plingeling) R 10040 10040 9630 0 -1 4202560 613 1252 0 0 145 0 0 0 25 0 5 0 526877172 127623168 3233 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10045/tid=10054] ppid=10040 vsize=124632 CPUtime=1.45 cores=4-7
/proc/10045/task/10054/stat : 10054 (plingeling) R 10040 10040 9630 0 -1 4202560 606 1252 0 0 145 0 0 0 25 0 5 0 526877172 127623168 3233 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331268 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10045/tid=10055] ppid=10040 vsize=124632 CPUtime=1.42 cores=4-7
/proc/10045/task/10055/stat : 10055 (plingeling) R 10040 10040 9630 0 -1 4202560 666 1252 0 0 142 0 0 0 25 0 5 0 526877172 127623168 3233 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.79
Current children cumulated vsize (KiB) 1472360

[startup+3.10084 s]
/proc/loadavg: 8.11 8.05 8.00 9/180 10055
/proc/meminfo: memFree=28960232/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=0 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526877168 18763776 389 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 4588166369492598783 0 0 17 6 0 0 0
/proc/10040/statm: 4581 389 341 23 0 91 0
[pid=10041] ppid=10040 vsize=860188 CPUtime=3.09 cores=0
/proc/10041/stat : 10041 (cryptominisat) R 10040 10040 9630 0 -1 4202496 9611 0 0 0 309 0 0 0 25 0 1 0 526877168 880832512 5294 33554432000 4194304 4587354 140733490335712 18446744073709551615 4364904 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10041/statm: 215047 5294 337 96 0 211609 0
[pid=10042] ppid=10040 vsize=19784 CPUtime=3.09 cores=1
/proc/10042/stat : 10042 (clasp) R 10040 10040 9630 0 -1 4202496 2421 0 0 0 309 0 0 0 25 0 1 0 526877168 20258816 2278 33554432000 4194304 4785740 140737258779488 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10042/statm: 4946 2278 341 145 0 1991 0
[pid=10043] ppid=10040 vsize=458072 CPUtime=3.09 cores=2
/proc/10043/stat : 10043 (TNM) R 10040 10040 9630 0 -1 4202496 1001 0 0 0 309 0 0 0 25 0 1 0 526877168 469065728 898 33554432000 134512640 134534564 4293247376 18446744073709551615 134528918 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10043/statm: 114518 898 95 6 0 113184 0
[pid=10044] ppid=10040 vsize=11768 CPUtime=3.09 cores=3
/proc/10044/stat : 10044 (march_hi) R 10040 10040 9630 0 -1 4202496 3827 0 0 0 308 1 0 0 25 0 1 0 526877168 12050432 1222 33554432000 4194304 4331988 140736650348672 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10044/statm: 2942 1222 174 34 0 1384 0
[pid=10045] ppid=10040 vsize=127420 CPUtime=12.22 cores=4-7
/proc/10045/stat : 10045 (plingeling) S 10040 10040 9630 0 -1 4202496 6259 1252 0 0 1221 1 0 0 25 0 5 0 526877168 130478080 3945 33554432000 4194304 4356188 140734915558016 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10045/statm: 31855 3945 162 40 0 30399 0
[pid=10045/tid=10052] ppid=10040 vsize=127420 CPUtime=3.05 cores=4-7
/proc/10045/task/10052/stat : 10052 (plingeling) R 10040 10040 9630 0 -1 4202560 793 1252 0 0 305 0 0 0 25 0 5 0 526877172 130478080 3945 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10045/tid=10053] ppid=10040 vsize=127420 CPUtime=3.05 cores=4-7
/proc/10045/task/10053/stat : 10053 (plingeling) R 10040 10040 9630 0 -1 4202560 1031 1252 0 0 305 0 0 0 25 0 5 0 526877172 130478080 3945 33554432000 4194304 4356188 140734915558016 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10045/tid=10054] ppid=10040 vsize=127420 CPUtime=3.05 cores=4-7
/proc/10045/task/10054/stat : 10054 (plingeling) R 10040 10040 9630 0 -1 4202560 784 1252 0 0 305 0 0 0 25 0 5 0 526877172 130478080 3945 33554432000 4194304 4356188 140734915558016 18446744073709551615 4238755 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10045/tid=10055] ppid=10040 vsize=127420 CPUtime=3.01 cores=4-7
/proc/10045/task/10055/stat : 10055 (plingeling) R 10040 10040 9630 0 -1 4202560 1117 1252 0 0 301 0 0 0 25 0 5 0 526877172 130478080 3945 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331050 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.58
Current children cumulated vsize (KiB) 1495556

[startup+6.30126 s]
/proc/loadavg: 8.10 8.05 8.00 9/180 10055
/proc/meminfo: memFree=28941340/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=0 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526877168 18763776 389 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 4588166369492598783 0 0 17 6 0 0 0
/proc/10040/statm: 4581 389 341 23 0 91 0
[pid=10041] ppid=10040 vsize=860184 CPUtime=6.28 cores=0
/proc/10041/stat : 10041 (cryptominisat) R 10040 10040 9630 0 -1 4202496 10687 0 0 0 628 0 0 0 25 0 1 0 526877168 880828416 5443 33554432000 4194304 4587354 140733490335712 18446744073709551615 4238611 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10041/statm: 215046 5443 337 96 0 211608 0
[pid=10042] ppid=10040 vsize=21612 CPUtime=6.29 cores=1
/proc/10042/stat : 10042 (clasp) R 10040 10040 9630 0 -1 4202496 2859 0 0 0 629 0 0 0 25 0 1 0 526877168 22130688 2716 33554432000 4194304 4785740 140737258779488 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10042/statm: 5403 2716 341 145 0 2448 0
[pid=10043] ppid=10040 vsize=458072 CPUtime=6.29 cores=2
/proc/10043/stat : 10043 (TNM) R 10040 10040 9630 0 -1 4202496 1001 0 0 0 629 0 0 0 25 0 1 0 526877168 469065728 898 33554432000 134512640 134534564 4293247376 18446744073709551615 3689476 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10043/statm: 114518 898 95 6 0 113184 0
[pid=10044] ppid=10040 vsize=11768 CPUtime=6.29 cores=3
/proc/10044/stat : 10044 (march_hi) R 10040 10040 9630 0 -1 4202496 3827 0 0 0 628 1 0 0 25 0 1 0 526877168 12050432 1222 33554432000 4194304 4331988 140736650348672 18446744073709551615 4299282 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10044/statm: 2942 1222 174 34 0 1384 0
[pid=10045] ppid=10040 vsize=128272 CPUtime=25.01 cores=4-7
/proc/10045/stat : 10045 (plingeling) S 10040 10040 9630 0 -1 4202496 6848 1252 0 0 2500 1 0 0 25 0 5 0 526877168 131350528 4150 33554432000 4194304 4356188 140734915558016 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10045/statm: 32068 4150 162 40 0 30612 0
[pid=10045/tid=10052] ppid=10040 vsize=128272 CPUtime=6.25 cores=4-7
/proc/10045/task/10052/stat : 10052 (plingeling) R 10040 10040 9630 0 -1 4202560 947 1252 0 0 625 0 0 0 25 0 5 0 526877172 131350528 4150 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331340 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10045/tid=10053] ppid=10040 vsize=128272 CPUtime=6.25 cores=4-7
/proc/10045/task/10053/stat : 10053 (plingeling) R 10040 10040 9630 0 -1 4202560 1111 1252 0 0 625 0 0 0 25 0 5 0 526877172 131350528 4150 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10045/tid=10054] ppid=10040 vsize=128272 CPUtime=6.25 cores=4-7
/proc/10045/task/10054/stat : 10054 (plingeling) R 10040 10040 9630 0 -1 4202560 930 1252 0 0 625 0 0 0 25 0 5 0 526877172 131350528 4150 33554432000 4194304 4356188 140734915558016 18446744073709551615 4270872 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10045/tid=10055] ppid=10040 vsize=128272 CPUtime=6.2 cores=4-7
/proc/10045/task/10055/stat : 10055 (plingeling) R 10040 10040 9630 0 -1 4202560 1326 1252 0 0 620 0 0 0 25 0 5 0 526877172 131350528 4150 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331819 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.16
Current children cumulated vsize (KiB) 1498232

[startup+12.7011 s]
/proc/loadavg: 8.09 8.05 8.00 9/180 10055
/proc/meminfo: memFree=28933616/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=0 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526877168 18763776 389 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 4588166369492598783 0 0 17 6 0 0 0
/proc/10040/statm: 4581 389 341 23 0 91 0
[pid=10041] ppid=10040 vsize=16302316 CPUtime=12.68 cores=0
/proc/10041/stat : 10041 (cryptominisat) R 10040 10040 9630 0 -1 4202496 15238 0 0 0 1267 1 0 0 25 0 1 0 526877168 16693571584 7720 33554432000 4194304 4587354 140733490335712 18446744073709551615 4388361 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10041/statm: 4075579 7720 337 96 0 4072141 0
[pid=10042] ppid=10040 vsize=24676 CPUtime=12.69 cores=1
/proc/10042/stat : 10042 (clasp) R 10040 10040 9630 0 -1 4202496 3560 0 0 0 1269 0 0 0 25 0 1 0 526877168 25268224 3417 33554432000 4194304 4785740 140737258779488 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10042/statm: 6169 3417 341 145 0 3214 0
[pid=10043] ppid=10040 vsize=458072 CPUtime=12.69 cores=2
/proc/10043/stat : 10043 (TNM) R 10040 10040 9630 0 -1 4202496 1001 0 0 0 1269 0 0 0 25 0 1 0 526877168 469065728 898 33554432000 134512640 134534564 4293247376 18446744073709551615 134530712 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10043/statm: 114518 898 95 6 0 113184 0
[pid=10044] ppid=10040 vsize=13304 CPUtime=12.69 cores=3
/proc/10044/stat : 10044 (march_hi) R 10040 10040 9630 0 -1 4202496 4134 0 0 0 1268 1 0 0 25 0 1 0 526877168 13623296 1529 33554432000 4194304 4331988 140736650348672 18446744073709551615 4246679 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10044/statm: 3326 1529 175 34 0 1768 0
[pid=10045] ppid=10040 vsize=128800 CPUtime=50.57 cores=4-7
/proc/10045/stat : 10045 (plingeling) S 10040 10040 9630 0 -1 4202496 7724 1252 0 0 5056 1 0 0 25 0 5 0 526877168 131891200 4290 33554432000 4194304 4356188 140734915558016 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10045/statm: 32200 4290 162 40 0 30744 0
[pid=10045/tid=10052] ppid=10040 vsize=128800 CPUtime=12.65 cores=4-7
/proc/10045/task/10052/stat : 10052 (plingeling) R 10040 10040 9630 0 -1 4202560 1212 1252 0 0 1265 0 0 0 25 0 5 0 526877172 131891200 4290 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10045/tid=10053] ppid=10040 vsize=128800 CPUtime=12.65 cores=4-7
/proc/10045/task/10053/stat : 10053 (plingeling) R 10040 10040 9630 0 -1 4202560 1402 1252 0 0 1265 0 0 0 25 0 5 0 526877172 131891200 4290 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10045/tid=10054] ppid=10040 vsize=128800 CPUtime=12.65 cores=4-7
/proc/10045/task/10054/stat : 10054 (plingeling) R 10040 10040 9630 0 -1 4202560 1122 1252 0 0 1265 0 0 0 25 0 5 0 526877172 131891200 4290 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331165 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10045/tid=10055] ppid=10040 vsize=128800 CPUtime=12.56 cores=4-7
/proc/10045/task/10055/stat : 10055 (plingeling) R 10040 10040 9630 0 -1 4202560 1454 1252 0 0 1256 0 0 0 25 0 5 0 526877172 131891200 4290 33554432000 4194304 4356188 140734915558016 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.32
Current children cumulated vsize (KiB) 16945492

[startup+25.5008 s]
/proc/loadavg: 7.99 8.03 8.00 8/179 10055
/proc/meminfo: memFree=28947956/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=19.09 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 522 17400 0 0 0 0 1907 2 15 0 1 0 526877168 18763776 390 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/10040/statm: 4581 390 342 23 0 91 0
[pid=10042] ppid=10040 vsize=28528 CPUtime=25.48 cores=1
/proc/10042/stat : 10042 (clasp) R 10040 10040 9630 0 -1 4202496 4436 0 0 0 2548 0 0 0 25 0 1 0 526877168 29212672 4293 33554432000 4194304 4785740 140737258779488 18446744073709551615 4287092 0 0 4096 16386 0 0 0 17 1 0 0 0

################
# More data... #
################

Current children cumulated CPU time (s) 33049.7
Current children cumulated vsize (KiB) 1511092

[startup+4782.3 s]
/proc/loadavg: 7.03 7.05 7.02 8/179 10194
/proc/meminfo: memFree=28234348/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=19.09 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 522 17400 0 0 0 0 1907 2 15 0 1 0 526877168 18763776 390 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/10040/statm: 4581 390 342 23 0 91 0
[pid=10042] ppid=10040 vsize=124320 CPUtime=4782.26 cores=1
/proc/10042/stat : 10042 (clasp) R 10040 10040 9630 0 -1 4202496 27515 0 0 0 478221 5 0 0 25 0 1 0 526877168 127303680 25282 33554432000 4194304 4785740 140737258779488 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10042/statm: 31080 25282 341 145 0 28125 0
[pid=10043] ppid=10040 vsize=458072 CPUtime=4781.97 cores=2
/proc/10043/stat : 10043 (TNM) R 10040 10040 9630 0 -1 4202496 1001 0 0 0 478192 5 0 0 25 0 1 0 526877168 469065728 898 33554432000 134512640 134534564 4293247376 18446744073709551615 134530598 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10043/statm: 114518 898 95 6 0 113184 0
[pid=10044] ppid=10040 vsize=691472 CPUtime=4782.42 cores=3
/proc/10044/stat : 10044 (march_hi) R 10040 10040 9630 0 -1 4202496 141335 0 0 0 478139 103 0 0 25 0 1 0 526877168 708067328 138730 33554432000 4194304 4331988 140736650348672 18446744073709551615 218909191164 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10044/statm: 172868 138730 175 34 0 171310 0
[pid=10045] ppid=10040 vsize=212812 CPUtime=19103.6 cores=4-7
/proc/10045/stat : 10045 (plingeling) S 10040 10040 9630 0 -1 4202496 501407 1252 0 0 1910278 79 0 0 25 0 5 0 526877168 217919488 25293 33554432000 4194304 4356188 140734915558016 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10045/statm: 53203 25293 162 40 0 51747 0
[pid=10045/tid=10052] ppid=10040 vsize=212812 CPUtime=4782.39 cores=4-7
/proc/10045/task/10052/stat : 10052 (plingeling) R 10040 10040 9630 0 -1 4202560 126369 1252 0 0 478220 19 0 0 25 0 5 0 526877172 217919488 25293 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331007 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10045/tid=10053] ppid=10040 vsize=212812 CPUtime=4782.38 cores=4-7
/proc/10045/task/10053/stat : 10053 (plingeling) R 10040 10040 9630 0 -1 4202560 123054 1252 0 0 478218 20 0 0 25 0 5 0 526877172 217919488 25293 33554432000 4194304 4356188 140734915558016 18446744073709551615 4332260 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10045/tid=10054] ppid=10040 vsize=212812 CPUtime=4782.36 cores=4-7
/proc/10045/task/10054/stat : 10054 (plingeling) R 10040 10040 9630 0 -1 4202560 119287 1252 0 0 478217 19 0 0 25 0 5 0 526877172 217919488 25293 33554432000 4194304 4356188 140734915558016 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10045/tid=10055] ppid=10040 vsize=212812 CPUtime=4756.35 cores=4-7
/proc/10045/task/10055/stat : 10055 (plingeling) R 10040 10040 9630 0 -1 4202560 130163 1252 0 0 475617 18 0 0 25 0 5 0 526877172 217919488 25293 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331007 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 33469.3
Current children cumulated vsize (KiB) 1505000

[startup+4842.3 s]
/proc/loadavg: 7.10 7.06 7.02 8/179 10196
/proc/meminfo: memFree=28198740/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=19.09 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 522 17400 0 0 0 0 1907 2 15 0 1 0 526877168 18763776 390 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/10040/statm: 4581 390 342 23 0 91 0
[pid=10042] ppid=10040 vsize=126360 CPUtime=4842.27 cores=1
/proc/10042/stat : 10042 (clasp) R 10040 10040 9630 0 -1 4202496 28011 0 0 0 484221 6 0 0 25 0 1 0 526877168 129392640 25706 33554432000 4194304 4785740 140737258779488 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10042/statm: 31590 25706 341 145 0 28635 0
[pid=10043] ppid=10040 vsize=458072 CPUtime=4841.96 cores=2
/proc/10043/stat : 10043 (TNM) R 10040 10040 9630 0 -1 4202496 1001 0 0 0 484191 5 0 0 25 0 1 0 526877168 469065728 898 33554432000 134512640 134534564 4293247376 18446744073709551615 134530606 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10043/statm: 114518 898 95 6 0 113184 0
[pid=10044] ppid=10040 vsize=691472 CPUtime=4842.43 cores=3
/proc/10044/stat : 10044 (march_hi) R 10040 10040 9630 0 -1 4202496 143104 0 0 0 484138 105 0 0 25 0 1 0 526877168 708067328 140499 33554432000 4194304 4331988 140736650348672 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10044/statm: 172868 140499 175 34 0 171310 0
[pid=10045] ppid=10040 vsize=241228 CPUtime=19343.2 cores=4-7
/proc/10045/stat : 10045 (plingeling) S 10040 10040 9630 0 -1 4202496 512607 1252 0 0 1934244 80 0 0 25 0 5 0 526877168 247017472 32397 33554432000 4194304 4356188 140734915558016 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10045/statm: 60307 32397 162 40 0 58851 0
[pid=10045/tid=10052] ppid=10040 vsize=241228 CPUtime=4842.39 cores=4-7
/proc/10045/task/10052/stat : 10052 (plingeling) R 10040 10040 9630 0 -1 4202560 128641 1252 0 0 484220 19 0 0 25 0 5 0 526877172 247017472 32397 33554432000 4194304 4356188 140734915558016 18446744073709551615 4298667 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10045/tid=10053] ppid=10040 vsize=241228 CPUtime=4842.39 cores=4-7
/proc/10045/task/10053/stat : 10053 (plingeling) R 10040 10040 9630 0 -1 4202560 126478 1252 0 0 484218 21 0 0 25 0 5 0 526877172 247017472 32397 33554432000 4194304 4356188 140734915558016 18446744073709551615 4330992 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10045/tid=10054] ppid=10040 vsize=241228 CPUtime=4842.36 cores=4-7
/proc/10045/task/10054/stat : 10054 (plingeling) R 10040 10040 9630 0 -1 4202560 122743 1252 0 0 484217 19 0 0 25 0 5 0 526877172 247017472 32397 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10045/tid=10055] ppid=10040 vsize=241228 CPUtime=4816.02 cores=4-7
/proc/10045/task/10055/stat : 10055 (plingeling) R 10040 10040 9630 0 -1 4202560 132211 1252 0 0 481584 18 0 0 25 0 5 0 526877172 247017472 32397 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 33889
Current children cumulated vsize (KiB) 1535456

[startup+4902.3 s]
/proc/loadavg: 7.15 7.08 7.03 8/179 10206
/proc/meminfo: memFree=28201284/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=19.09 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 522 17400 0 0 0 0 1907 2 15 0 1 0 526877168 18763776 390 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/10040/statm: 4581 390 342 23 0 91 0
[pid=10042] ppid=10040 vsize=127504 CPUtime=4902.27 cores=1
/proc/10042/stat : 10042 (clasp) R 10040 10040 9630 0 -1 4202496 28266 0 0 0 490221 6 0 0 25 0 1 0 526877168 130564096 25961 33554432000 4194304 4785740 140737258779488 18446744073709551615 4286927 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10042/statm: 31876 25961 341 145 0 28921 0
[pid=10043] ppid=10040 vsize=458072 CPUtime=4901.96 cores=2
/proc/10043/stat : 10043 (TNM) R 10040 10040 9630 0 -1 4202496 1001 0 0 0 490191 5 0 0 25 0 1 0 526877168 469065728 898 33554432000 134512640 134534564 4293247376 18446744073709551615 134528680 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10043/statm: 114518 898 95 6 0 113184 0
[pid=10044] ppid=10040 vsize=691472 CPUtime=4902.43 cores=3
/proc/10044/stat : 10044 (march_hi) R 10040 10040 9630 0 -1 4202496 144891 0 0 0 490137 106 0 0 25 0 1 0 526877168 708067328 142286 33554432000 4194304 4331988 140736650348672 18446744073709551615 4210430 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10044/statm: 172868 142286 175 34 0 171310 0
[pid=10045] ppid=10040 vsize=229580 CPUtime=19582.9 cores=4-7
/proc/10045/stat : 10045 (plingeling) S 10040 10040 9630 0 -1 4202496 515007 1252 0 0 1958212 80 0 0 25 0 5 0 526877168 235089920 29485 33554432000 4194304 4356188 140734915558016 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10045/statm: 57395 29485 162 40 0 55939 0
[pid=10045/tid=10052] ppid=10040 vsize=229580 CPUtime=4902.39 cores=4-7
/proc/10045/task/10052/stat : 10052 (plingeling) R 10040 10040 9630 0 -1 4202560 130049 1252 0 0 490220 19 0 0 25 0 5 0 526877172 235089920 29485 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10045/tid=10053] ppid=10040 vsize=229580 CPUtime=4902.39 cores=4-7
/proc/10045/task/10053/stat : 10053 (plingeling) R 10040 10040 9630 0 -1 4202560 126734 1252 0 0 490218 21 0 0 25 0 5 0 526877172 235089920 29485 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10045/tid=10054] ppid=10040 vsize=229580 CPUtime=4902.37 cores=4-7
/proc/10045/task/10054/stat : 10054 (plingeling) R 10040 10040 9630 0 -1 4202560 122871 1252 0 0 490217 20 0 0 25 0 5 0 526877172 235089920 29485 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10045/tid=10055] ppid=10040 vsize=229580 CPUtime=4875.71 cores=4-7
/proc/10045/task/10055/stat : 10055 (plingeling) R 10040 10040 9630 0 -1 4202560 132819 1252 0 0 487552 19 0 0 25 0 5 0 526877172 235089920 29485 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331045 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 34308.7
Current children cumulated vsize (KiB) 1524952

[startup+4962.3 s]
/proc/loadavg: 7.09 7.08 7.02 8/179 10208
/proc/meminfo: memFree=28200636/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=19.09 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 522 17400 0 0 0 0 1907 2 15 0 1 0 526877168 18763776 390 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/10040/statm: 4581 390 342 23 0 91 0
[pid=10042] ppid=10040 vsize=128216 CPUtime=4962.27 cores=1
/proc/10042/stat : 10042 (clasp) R 10040 10040 9630 0 -1 4202496 28448 0 0 0 496221 6 0 0 25 0 1 0 526877168 131293184 26143 33554432000 4194304 4785740 140737258779488 18446744073709551615 4287158 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10042/statm: 32054 26143 341 145 0 29099 0
[pid=10043] ppid=10040 vsize=458072 CPUtime=4961.96 cores=2
/proc/10043/stat : 10043 (TNM) R 10040 10040 9630 0 -1 4202496 1001 0 0 0 496190 6 0 0 25 0 1 0 526877168 469065728 898 33554432000 134512640 134534564 4293247376 18446744073709551615 134528708 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10043/statm: 114518 898 95 6 0 113184 0
[pid=10044] ppid=10040 vsize=691472 CPUtime=4962.44 cores=3
/proc/10044/stat : 10044 (march_hi) R 10040 10040 9630 0 -1 4202496 146557 0 0 0 496136 108 0 0 25 0 1 0 526877168 708067328 143952 33554432000 4194304 4331988 140736650348672 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10044/statm: 172868 143952 175 34 0 171310 0
[pid=10045] ppid=10040 vsize=221516 CPUtime=19822.6 cores=4-7
/proc/10045/stat : 10045 (plingeling) S 10040 10040 9630 0 -1 4202496 518303 1252 0 0 1982179 81 0 0 25 0 5 0 526877168 226832384 27469 33554432000 4194304 4356188 140734915558016 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10045/statm: 55379 27469 162 40 0 53923 0
[pid=10045/tid=10052] ppid=10040 vsize=221516 CPUtime=4962.39 cores=4-7
/proc/10045/task/10052/stat : 10052 (plingeling) R 10040 10040 9630 0 -1 4202560 130049 1252 0 0 496220 19 0 0 25 0 5 0 526877172 226832384 27469 33554432000 4194304 4356188 140734915558016 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10045/tid=10053] ppid=10040 vsize=221516 CPUtime=4962.39 cores=4-7
/proc/10045/task/10053/stat : 10053 (plingeling) R 10040 10040 9630 0 -1 4202560 128782 1252 0 0 496218 21 0 0 25 0 5 0 526877172 226832384 27469 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331002 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10045/tid=10054] ppid=10040 vsize=221516 CPUtime=4962.37 cores=4-7
/proc/10045/task/10054/stat : 10054 (plingeling) R 10040 10040 9630 0 -1 4202560 123095 1252 0 0 496217 20 0 0 25 0 5 0 526877172 226832384 27469 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331960 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10045/tid=10055] ppid=10040 vsize=221516 CPUtime=4935.38 cores=4-7
/proc/10045/task/10055/stat : 10055 (plingeling) R 10040 10040 9630 0 -1 4202560 133843 1252 0 0 493519 19 0 0 25 0 5 0 526877172 226832384 27469 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331062 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 34728.4
Current children cumulated vsize (KiB) 1517600



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 7.11 7.08 7.03 8/179 10209
/proc/meminfo: memFree=28205324/32951124 swapFree=67111356/67111528
[pid=10040] ppid=10038 vsize=18324 CPUtime=19.09 cores=0-7
/proc/10040/stat : 10040 (ppfolio) S 10038 10040 9630 0 -1 4202496 522 17400 0 0 0 0 1907 2 15 0 1 0 526877168 18763776 390 33554432000 4194304 4287625 140734706520848 18446744073709551615 218905751055 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/10040/statm: 4581 390 342 23 0 91 0
[pid=10042] ppid=10040 vsize=128500 CPUtime=4999.98 cores=1
/proc/10042/stat : 10042 (clasp) R 10040 10040 9630 0 -1 4202496 28507 0 0 0 499992 6 0 0 25 0 1 0 526877168 131584000 26202 33554432000 4194304 4785740 140737258779488 18446744073709551615 4287013 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10042/statm: 32125 26202 341 145 0 29170 0
[pid=10043] ppid=10040 vsize=458072 CPUtime=4999.66 cores=2
/proc/10043/stat : 10043 (TNM) R 10040 10040 9630 0 -1 4202496 1001 0 0 0 499960 6 0 0 25 0 1 0 526877168 469065728 898 33554432000 134512640 134534564 4293247376 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10043/statm: 114518 898 95 6 0 113184 0
[pid=10044] ppid=10040 vsize=691472 CPUtime=5000.14 cores=3
/proc/10044/stat : 10044 (march_hi) R 10040 10040 9630 0 -1 4202496 147651 0 0 0 499905 109 0 0 25 0 1 0 526877168 708067328 145046 33554432000 4194304 4331988 140736650348672 18446744073709551615 4239602 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10044/statm: 172868 145046 175 34 0 171310 0
[pid=10045] ppid=10040 vsize=211660 CPUtime=19973.2 cores=4-7
/proc/10045/stat : 10045 (plingeling) S 10040 10040 9630 0 -1 4202496 519935 1252 0 0 1997240 82 0 0 25 0 5 0 526877168 216739840 25005 33554432000 4194304 4356188 140734915558016 18446744073709551615 218917533669 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/10045/statm: 52915 25005 162 40 0 51459 0
[pid=10045/tid=10052] ppid=10040 vsize=211660 CPUtime=5000.1 cores=4-7
/proc/10045/task/10052/stat : 10052 (plingeling) R 10040 10040 9630 0 -1 4202560 130049 1252 0 0 499991 19 0 0 25 0 5 0 526877172 216739840 25005 33554432000 4194304 4356188 140734915558016 18446744073709551615 4332275 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10045/tid=10053] ppid=10040 vsize=211660 CPUtime=5000.1 cores=4-7
/proc/10045/task/10053/stat : 10053 (plingeling) R 10040 10040 9630 0 -1 4202560 129006 1252 0 0 499988 22 0 0 25 0 5 0 526877172 216739840 25005 33554432000 4194304 4356188 140734915558016 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10045/tid=10054] ppid=10040 vsize=211660 CPUtime=5000.08 cores=4-7
/proc/10045/task/10054/stat : 10054 (plingeling) R 10040 10040 9630 0 -1 4202560 124503 1252 0 0 499988 20 0 0 25 0 5 0 526877172 216739840 25005 33554432000 4194304 4356188 140734915558016 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10045/tid=10055] ppid=10040 vsize=211660 CPUtime=4972.88 cores=4-7
/proc/10045/task/10055/stat : 10055 (plingeling) R 10040 10040 9630 0 -1 4202560 133843 1252 0 0 497269 19 0 0 25 0 5 0 526877172 216739840 25005 33554432000 4194304 4356188 140734915558016 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 34992.1
Current children cumulated vsize (KiB) 1508028

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 10040 and gives
#  childrusage.ru_utime.tv_sec=10018
#  childrusage.ru_utime.tv_usec=602940
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=168974
# CPU time returned by wait4() is 10018.8
# while last known CPU time is 34992.1
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 34992.1
CPU user time (s): 34990
CPU system time (s): 2.05
CPU usage (%): 699.839
Max. virtual memory (cumulated for all children) (KiB): 27466604

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 10018.6
system time used= 0.168974
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 47445
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 450
involuntary context switches= 35106

runsolver used 8.16876 second user time and 19.0361 second system time

The end

Launcher Data 

Begin job on node137 at 2011-04-30 10:39:55
IDJOB=3219729
IDBENCH=82638
IDSOLVER=1638
FILE ID=node137/3219729-1304152794
RUNJOBID= node137-1304145473-9644
PBS_JOBID= 13170887
Free space on /tmp= 73212 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k7-r85-v200-c17000-S970203146-047-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3219729-1304152794/watcher-3219729-1304152794 -o /tmp/evaluation-result-3219729-1304152794/solver-3219729-1304152794 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3219729-1304152794.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 635ea520fe16a78d41337039f98997e5
RANDOM SEED=1931248290

node137.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      28992584 kB
Buffers:        919676 kB
Cached:        2517016 kB
SwapCached:         48 kB
Active:         194524 kB
Inactive:      3305472 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      28992584 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:           10480 kB
Writeback:           0 kB
AnonPages:       64224 kB
Mapped:          15152 kB
Slab:           392104 kB
PageTables:       5340 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   186340 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73200 MiB
End job on node137 at 2011-04-30 12:03:17