Trace number 3222417

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39971.4 5000.02

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v80-c7120-S1968028041-077.UNKNOWN.cnf
MD5SUMb69cb091f87b56cc59075b29ac9c632c
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables80
Number of clauses7120
Sum of the clauses size49840
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 57120

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3222417-1304165602/watcher-3222417-1304165602 -o /tmp/evaluation-result-3222417-1304165602/solver-3222417-1304165602 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3222417-1304165602.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.49 7.66 7.88 1/170 31274
/proc/meminfo: memFree=30971848/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 25 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 6 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0

[startup+0.0687331 s]
/proc/loadavg: 6.49 7.66 7.88 1/170 31274
/proc/meminfo: memFree=30971848/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 25 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.168714 s]
/proc/loadavg: 6.49 7.66 7.88 1/170 31274
/proc/meminfo: memFree=30971848/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 25 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.368678 s]
/proc/loadavg: 6.49 7.66 7.88 1/170 31274
/proc/meminfo: memFree=30971848/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 23 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.70062 s]
/proc/loadavg: 6.49 7.66 7.88 1/170 31274
/proc/meminfo: memFree=30971848/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) R 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 21 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50048 s]
/proc/loadavg: 6.69 7.68 7.88 10/180 31289
/proc/meminfo: memFree=30951432/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
[pid=31275] ppid=31274 vsize=908916 CPUtime=1.49 cores=0
/proc/31275/stat : 31275 (cryptominisat) R 31274 31274 30879 0 -1 4202496 4623 0 0 0 149 0 0 0 20 0 1 0 508976455 930729984 2783 33554432000 4194304 4587354 140737139711552 18446744073709551615 4388484 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31275/statm: 227229 2783 336 96 0 223791 0
[pid=31276] ppid=31274 vsize=15024 CPUtime=1.49 cores=1
/proc/31276/stat : 31276 (clasp) R 31274 31274 30879 0 -1 4202496 1207 0 0 0 149 0 0 0 20 0 1 0 508976455 15384576 1099 33554432000 4194304 4785740 140735201266704 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31276/statm: 3756 1099 340 145 0 801 0
[pid=31277] ppid=31274 vsize=456648 CPUtime=1.49 cores=2
/proc/31277/stat : 31277 (TNM) R 31274 31274 30879 0 -1 4202496 555 0 0 0 149 0 0 0 20 0 1 0 508976455 467607552 452 33554432000 134512640 134534564 4291756704 18446744073709551615 134530593 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31277/statm: 114162 452 95 6 0 112828 0
[pid=31278] ppid=31274 vsize=8580 CPUtime=1.49 cores=3
/proc/31278/stat : 31278 (march_hi) R 31274 31274 30879 0 -1 4202496 1555 0 0 0 149 0 0 0 20 0 1 0 508976455 8785920 627 33554432000 4194304 4331988 140733648135968 18446744073709551615 4226260 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31278/statm: 2145 627 174 34 0 587 0
[pid=31279] ppid=31274 vsize=126980 CPUtime=5.91 cores=4-7
/proc/31279/stat : 31279 (plingeling) S 31274 31274 30879 0 -1 4202496 4819 1252 0 0 591 0 0 0 18 0 5 0 508976455 130027520 3820 33554432000 4194304 4356188 140736612774608 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31279/statm: 31745 3820 162 40 0 30289 0
[pid=31279/tid=31286] ppid=31274 vsize=126980 CPUtime=1.47 cores=4-7
/proc/31279/task/31286/stat : 31286 (plingeling) R 31274 31274 30879 0 -1 4202560 979 1252 0 0 147 0 0 0 23 0 5 0 508976457 130027520 3820 33554432000 4194304 4356188 140736612774608 18446744073709551615 4279467 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31279/tid=31287] ppid=31274 vsize=126980 CPUtime=1.48 cores=4-7
/proc/31279/task/31287/stat : 31287 (plingeling) R 31274 31274 30879 0 -1 4202560 916 1252 0 0 148 0 0 0 23 0 5 0 508976457 130027520 3820 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331984 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31279/tid=31288] ppid=31274 vsize=126980 CPUtime=1.47 cores=4-7
/proc/31279/task/31288/stat : 31288 (plingeling) R 31274 31274 30879 0 -1 4202560 914 1252 0 0 147 0 0 0 23 0 5 0 508976457 130027520 3820 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331151 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31279/tid=31289] ppid=31274 vsize=126980 CPUtime=1.48 cores=4-7
/proc/31279/task/31289/stat : 31289 (plingeling) R 31274 31274 30879 0 -1 4202560 821 1252 0 0 148 0 0 0 23 0 5 0 508976457 130027520 3820 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.87
Current children cumulated vsize (KiB) 1534472

[startup+3.10119 s]
/proc/loadavg: 6.69 7.68 7.88 9/180 31289
/proc/meminfo: memFree=30939156/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
[pid=31275] ppid=31274 vsize=34916 CPUtime=3.08 cores=0
/proc/31275/stat : 31275 (cryptominisat) R 31274 31274 30879 0 -1 4202496 5784 0 0 0 308 0 0 0 25 0 1 0 508976455 35753984 2936 33554432000 4194304 4587354 140737139711552 18446744073709551615 4238618 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31275/statm: 8729 2936 336 96 0 5291 0
[pid=31276] ppid=31274 vsize=15884 CPUtime=3.09 cores=1
/proc/31276/stat : 31276 (clasp) R 31274 31274 30879 0 -1 4202496 1413 0 0 0 309 0 0 0 25 0 1 0 508976455 16265216 1305 33554432000 4194304 4785740 140735201266704 18446744073709551615 4287050 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31276/statm: 3971 1305 340 145 0 1016 0
[pid=31277] ppid=31274 vsize=456648 CPUtime=3.08 cores=2
/proc/31277/stat : 31277 (TNM) R 31274 31274 30879 0 -1 4202496 555 0 0 0 308 0 0 0 25 0 1 0 508976455 467607552 452 33554432000 134512640 134534564 4291756704 18446744073709551615 134528275 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31277/statm: 114162 452 95 6 0 112828 0
[pid=31278] ppid=31274 vsize=8884 CPUtime=3.09 cores=3
/proc/31278/stat : 31278 (march_hi) R 31274 31274 30879 0 -1 4202496 1628 0 0 0 309 0 0 0 25 0 1 0 508976455 9097216 700 33554432000 4194304 4331988 140733648135968 18446744073709551615 4299339 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31278/statm: 2221 700 174 34 0 663 0
[pid=31279] ppid=31274 vsize=127000 CPUtime=12.31 cores=4-7
/proc/31279/stat : 31279 (plingeling) S 31274 31274 30879 0 -1 4202496 5563 1252 0 0 1231 0 0 0 18 0 5 0 508976455 130048000 3828 33554432000 4194304 4356188 140736612774608 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31279/statm: 31750 3828 162 40 0 30294 0
[pid=31279/tid=31286] ppid=31274 vsize=127000 CPUtime=3.06 cores=4-7
/proc/31279/task/31286/stat : 31286 (plingeling) R 31274 31274 30879 0 -1 4202560 1143 1252 0 0 306 0 0 0 25 0 5 0 508976457 130048000 3828 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331151 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31279/tid=31287] ppid=31274 vsize=127000 CPUtime=3.08 cores=4-7
/proc/31279/task/31287/stat : 31287 (plingeling) R 31274 31274 30879 0 -1 4202560 1077 1252 0 0 308 0 0 0 25 0 5 0 508976457 130048000 3828 33554432000 4194304 4356188 140736612774608 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31279/tid=31288] ppid=31274 vsize=127000 CPUtime=3.07 cores=4-7
/proc/31279/task/31288/stat : 31288 (plingeling) R 31274 31274 30879 0 -1 4202560 1043 1252 0 0 307 0 0 0 25 0 5 0 508976457 130048000 3828 33554432000 4194304 4356188 140736612774608 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31279/tid=31289] ppid=31274 vsize=127000 CPUtime=3.08 cores=4-7
/proc/31279/task/31289/stat : 31289 (plingeling) R 31274 31274 30879 0 -1 4202560 1111 1252 0 0 308 0 0 0 25 0 5 0 508976457 130048000 3828 33554432000 4194304 4356188 140736612774608 18446744073709551615 4281793 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.65
Current children cumulated vsize (KiB) 661656

[startup+6.30062 s]
/proc/loadavg: 6.79 7.69 7.88 9/180 31289
/proc/meminfo: memFree=30935940/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
[pid=31275] ppid=31274 vsize=915100 CPUtime=6.29 cores=0
/proc/31275/stat : 31275 (cryptominisat) R 31274 31274 30879 0 -1 4202496 7489 0 0 0 628 1 0 0 25 0 1 0 508976455 937062400 3460 33554432000 4194304 4587354 140737139711552 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31275/statm: 228775 3460 336 96 0 225337 0
[pid=31276] ppid=31274 vsize=16984 CPUtime=6.29 cores=1
/proc/31276/stat : 31276 (clasp) R 31274 31274 30879 0 -1 4202496 1655 0 0 0 629 0 0 0 25 0 1 0 508976455 17391616 1547 33554432000 4194304 4785740 140735201266704 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31276/statm: 4246 1547 340 145 0 1291 0
[pid=31277] ppid=31274 vsize=456648 CPUtime=6.28 cores=2
/proc/31277/stat : 31277 (TNM) R 31274 31274 30879 0 -1 4202496 555 0 0 0 628 0 0 0 25 0 1 0 508976455 467607552 452 33554432000 134512640 134534564 4291756704 18446744073709551615 134528888 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31277/statm: 114162 452 95 6 0 112828 0
[pid=31278] ppid=31274 vsize=9264 CPUtime=6.29 cores=3
/proc/31278/stat : 31278 (march_hi) R 31274 31274 30879 0 -1 4202496 1721 0 0 0 629 0 0 0 25 0 1 0 508976455 9486336 793 33554432000 4194304 4331988 140733648135968 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31278/statm: 2316 793 175 34 0 758 0
[pid=31279] ppid=31274 vsize=128360 CPUtime=25.1 cores=4-7
/proc/31279/stat : 31279 (plingeling) S 31274 31274 30879 0 -1 4202496 7206 1252 0 0 2509 1 0 0 18 0 5 0 508976455 131440640 4175 33554432000 4194304 4356188 140736612774608 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31279/statm: 32090 4175 162 40 0 30634 0
[pid=31279/tid=31286] ppid=31274 vsize=128360 CPUtime=6.24 cores=4-7
/proc/31279/task/31286/stat : 31286 (plingeling) R 31274 31274 30879 0 -1 4202560 1470 1252 0 0 624 0 0 0 25 0 5 0 508976457 131440640 4175 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331151 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31279/tid=31287] ppid=31274 vsize=128360 CPUtime=6.28 cores=4-7
/proc/31279/task/31287/stat : 31287 (plingeling) R 31274 31274 30879 0 -1 4202560 1526 1252 0 0 628 0 0 0 25 0 5 0 508976457 131440640 4175 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31279/tid=31288] ppid=31274 vsize=128360 CPUtime=6.27 cores=4-7
/proc/31279/task/31288/stat : 31288 (plingeling) R 31274 31274 30879 0 -1 4202560 1461 1252 0 0 627 0 0 0 25 0 5 0 508976457 131440640 4175 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331193 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31279/tid=31289] ppid=31274 vsize=128360 CPUtime=6.28 cores=4-7
/proc/31279/task/31289/stat : 31289 (plingeling) R 31274 31274 30879 0 -1 4202560 1560 1252 0 0 628 0 0 0 25 0 5 0 508976457 131440640 4175 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331576 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.25
Current children cumulated vsize (KiB) 1544680

[startup+12.7005 s]
/proc/loadavg: 6.89 7.69 7.89 9/180 31290
/proc/meminfo: memFree=30921828/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
[pid=31275] ppid=31274 vsize=918640 CPUtime=12.68 cores=0
/proc/31275/stat : 31275 (cryptominisat) R 31274 31274 30879 0 -1 4202496 12281 0 0 0 1267 1 0 0 25 0 1 0 508976455 940687360 5360 33554432000 4194304 4587354 140737139711552 18446744073709551615 4268411 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31275/statm: 229660 5360 336 96 0 226222 0
[pid=31276] ppid=31274 vsize=18748 CPUtime=12.69 cores=1
/proc/31276/stat : 31276 (clasp) R 31274 31274 30879 0 -1 4202496 2057 0 0 0 1269 0 0 0 25 0 1 0 508976455 19197952 1949 33554432000 4194304 4785740 140735201266704 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31276/statm: 4687 1949 340 145 0 1732 0
[pid=31277] ppid=31274 vsize=456648 CPUtime=12.68 cores=2
/proc/31277/stat : 31277 (TNM) R 31274 31274 30879 0 -1 4202496 555 0 0 0 1268 0 0 0 25 0 1 0 508976455 467607552 452 33554432000 134512640 134534564 4291756704 18446744073709551615 134529062 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31277/statm: 114162 452 95 6 0 112828 0
[pid=31278] ppid=31274 vsize=10116 CPUtime=12.69 cores=3
/proc/31278/stat : 31278 (march_hi) R 31274 31274 30879 0 -1 4202496 1900 0 0 0 1269 0 0 0 25 0 1 0 508976455 10358784 972 33554432000 4194304 4331988 140733648135968 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31278/statm: 2529 972 175 34 0 971 0
[pid=31279] ppid=31274 vsize=135416 CPUtime=50.65 cores=4-7
/proc/31279/stat : 31279 (plingeling) S 31274 31274 30879 0 -1 4202496 11630 1252 0 0 5064 1 0 0 18 0 5 0 508976455 138665984 5943 33554432000 4194304 4356188 140736612774608 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31279/statm: 33854 5943 162 40 0 32398 0
[pid=31279/tid=31286] ppid=31274 vsize=135416 CPUtime=12.61 cores=4-7
/proc/31279/task/31286/stat : 31286 (plingeling) R 31274 31274 30879 0 -1 4202560 2307 1252 0 0 1261 0 0 0 25 0 5 0 508976457 138665984 5943 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331086 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31279/tid=31287] ppid=31274 vsize=135416 CPUtime=12.67 cores=4-7
/proc/31279/task/31287/stat : 31287 (plingeling) R 31274 31274 30879 0 -1 4202560 2968 1252 0 0 1267 0 0 0 25 0 5 0 508976457 138665984 5943 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31279/tid=31288] ppid=31274 vsize=135416 CPUtime=12.67 cores=4-7
/proc/31279/task/31288/stat : 31288 (plingeling) R 31274 31274 30879 0 -1 4202560 2229 1252 0 0 1267 0 0 0 25 0 5 0 508976457 138665984 5943 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31279/tid=31289] ppid=31274 vsize=135416 CPUtime=12.67 cores=4-7
/proc/31279/task/31289/stat : 31289 (plingeling) R 31274 31274 30879 0 -1 4202560 2937 1252 0 0 1267 0 0 0 25 0 5 0 508976457 138665984 5943 33554432000 4194304 4356188 140736612774608 18446744073709551615 4238504 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.39
Current children cumulated vsize (KiB) 1557892

[startup+25.5012 s]
/proc/loadavg: 7.13 7.71 7.89 9/180 31290
/proc/meminfo: memFree=30905020/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
[pid=31275] ppid=31274 vsize=925324 CPUtime=25.49 cores=0
/proc/31275/stat : 31275 (cryptominisat) R 31274 31274 30879 0 -1 4202496 18627 0 0 0 2546 3 0 0 25 0 1 0 508976455 947531776 7160 33554432000 4194304 4587354 140737139711552 18446744073709551615 4388356 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=31277] ppid=31274 vsize=456648 CPUtime=4781.81 cores=2
/proc/31277/stat : 31277 (TNM) R 31274 31274 30879 0 -1 4202496 555 0 0 0 478179 2 0 0 25 0 1 0 508976455 467607552 452 33554432000 134512640 134534564 4291756704 18446744073709551615 134530504 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31277/statm: 114162 452 95 6 0 112828 0
[pid=31278] ppid=31274 vsize=464356 CPUtime=4782.41 cores=3
/proc/31278/stat : 31278 (march_hi) R 31274 31274 30879 0 -1 4202496 112620 0 0 0 478158 83 0 0 25 0 1 0 508976455 475500544 111692 33554432000 4194304 4331988 140733648135968 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31278/statm: 116089 111692 175 34 0 114531 0
[pid=31279] ppid=31274 vsize=324132 CPUtime=19101.9 cores=4-7
/proc/31279/stat : 31279 (plingeling) S 31274 31274 30879 0 -1 4202496 594612 1252 0 0 1910106 87 0 0 18 0 5 0 508976455 331911168 53139 33554432000 4194304 4356188 140736612774608 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31279/statm: 81033 53139 162 40 0 79577 0
[pid=31279/tid=31286] ppid=31274 vsize=324132 CPUtime=4754.7 cores=4-7
/proc/31279/task/31286/stat : 31286 (plingeling) R 31274 31274 30879 0 -1 4202560 150815 1252 0 0 475450 20 0 0 25 0 5 0 508976457 331911168 53139 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31279/tid=31287] ppid=31274 vsize=324132 CPUtime=4782.4 cores=4-7
/proc/31279/task/31287/stat : 31287 (plingeling) R 31274 31274 30879 0 -1 4202560 145595 1252 0 0 478218 22 0 0 25 0 5 0 508976457 331911168 53139 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31279/tid=31288] ppid=31274 vsize=324132 CPUtime=4782.39 cores=4-7
/proc/31279/task/31288/stat : 31288 (plingeling) R 31274 31274 30879 0 -1 4202560 145689 1252 0 0 478216 23 0 0 25 0 5 0 508976457 331911168 53139 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31279/tid=31289] ppid=31274 vsize=324132 CPUtime=4782.4 cores=4-7
/proc/31279/task/31289/stat : 31289 (plingeling) R 31274 31274 30879 0 -1 4202560 151324 1252 0 0 478220 20 0 0 25 0 5 0 508976457 331911168 53139 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331264 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38231
Current children cumulated vsize (KiB) 1547820

[startup+4842.3 s]
/proc/loadavg: 8.06 8.07 8.01 9/180 31433
/proc/meminfo: memFree=30146432/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
[pid=31275] ppid=31274 vsize=206732 CPUtime=4842.41 cores=0
/proc/31275/stat : 31275 (cryptominisat) R 31274 31274 30879 0 -1 4202496 289573 0 0 0 484115 126 0 0 25 0 1 0 508976455 211693568 29604 33554432000 4194304 4587354 140737139711552 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31275/statm: 51683 29604 336 96 0 48245 0
[pid=31276] ppid=31274 vsize=59132 CPUtime=4842.4 cores=1
/proc/31276/stat : 31276 (clasp) R 31274 31274 30879 0 -1 4202496 12964 0 0 0 484238 2 0 0 25 0 1 0 508976455 60551168 10468 33554432000 4194304 4785740 140735201266704 18446744073709551615 4287158 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31276/statm: 14783 10468 341 145 0 11828 0
[pid=31277] ppid=31274 vsize=456648 CPUtime=4841.81 cores=2
/proc/31277/stat : 31277 (TNM) R 31274 31274 30879 0 -1 4202496 555 0 0 0 484179 2 0 0 25 0 1 0 508976455 467607552 452 33554432000 134512640 134534564 4291756704 18446744073709551615 134528295 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31277/statm: 114162 452 95 6 0 112828 0
[pid=31278] ppid=31274 vsize=472600 CPUtime=4842.41 cores=3
/proc/31278/stat : 31278 (march_hi) R 31274 31274 30879 0 -1 4202496 113956 0 0 0 484156 85 0 0 25 0 1 0 508976455 483942400 113028 33554432000 4194304 4331988 140733648135968 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31278/statm: 118150 113028 175 34 0 116592 0
[pid=31279] ppid=31274 vsize=326948 CPUtime=19341.6 cores=4-7
/proc/31279/stat : 31279 (plingeling) S 31274 31274 30879 0 -1 4202496 599860 1252 0 0 1934073 88 0 0 18 0 5 0 508976455 334794752 53843 33554432000 4194304 4356188 140736612774608 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31279/statm: 81737 53843 162 40 0 80281 0
[pid=31279/tid=31286] ppid=31274 vsize=326948 CPUtime=4814.37 cores=4-7
/proc/31279/task/31286/stat : 31286 (plingeling) R 31274 31274 30879 0 -1 4202560 153055 1252 0 0 481417 20 0 0 25 0 5 0 508976457 334794752 53843 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331589 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31279/tid=31287] ppid=31274 vsize=326948 CPUtime=4842.4 cores=4-7
/proc/31279/task/31287/stat : 31287 (plingeling) R 31274 31274 30879 0 -1 4202560 146811 1252 0 0 484218 22 0 0 25 0 5 0 508976457 334794752 53843 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31279/tid=31288] ppid=31274 vsize=326948 CPUtime=4842.39 cores=4-7
/proc/31279/task/31288/stat : 31288 (plingeling) R 31274 31274 30879 0 -1 4202560 147481 1252 0 0 484216 23 0 0 25 0 5 0 508976457 334794752 53843 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31279/tid=31289] ppid=31274 vsize=326948 CPUtime=4842.4 cores=4-7
/proc/31279/task/31289/stat : 31289 (plingeling) R 31274 31274 30879 0 -1 4202560 151324 1252 0 0 484220 20 0 0 25 0 5 0 508976457 334794752 53843 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38710.6
Current children cumulated vsize (KiB) 1540384

[startup+4902.3 s]
/proc/loadavg: 8.05 8.07 8.01 9/180 31435
/proc/meminfo: memFree=30130504/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
[pid=31275] ppid=31274 vsize=151088 CPUtime=4902.42 cores=0
/proc/31275/stat : 31275 (cryptominisat) R 31274 31274 30879 0 -1 4202496 293741 0 0 0 490114 128 0 0 25 0 1 0 508976455 154714112 32436 33554432000 4194304 4587354 140737139711552 18446744073709551615 4238752 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31275/statm: 37772 32436 336 96 0 34334 0
[pid=31276] ppid=31274 vsize=59132 CPUtime=4902.4 cores=1
/proc/31276/stat : 31276 (clasp) R 31274 31274 30879 0 -1 4202496 12968 0 0 0 490238 2 0 0 25 0 1 0 508976455 60551168 10472 33554432000 4194304 4785740 140735201266704 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31276/statm: 14783 10472 341 145 0 11828 0
[pid=31277] ppid=31274 vsize=456648 CPUtime=4901.8 cores=2
/proc/31277/stat : 31277 (TNM) R 31274 31274 30879 0 -1 4202496 555 0 0 0 490178 2 0 0 25 0 1 0 508976455 467607552 452 33554432000 134512640 134534564 4291756704 18446744073709551615 134528913 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31277/statm: 114162 452 95 6 0 112828 0
[pid=31278] ppid=31274 vsize=472600 CPUtime=4902.42 cores=3
/proc/31278/stat : 31278 (march_hi) R 31274 31274 30879 0 -1 4202496 115222 0 0 0 490156 86 0 0 25 0 1 0 508976455 483942400 114294 33554432000 4194304 4331988 140733648135968 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31278/statm: 118150 114294 175 34 0 116592 0
[pid=31279] ppid=31274 vsize=327460 CPUtime=19581.3 cores=4-7
/proc/31279/stat : 31279 (plingeling) S 31274 31274 30879 0 -1 4202496 604468 1252 0 0 1958041 88 0 0 18 0 5 0 508976455 335319040 53971 33554432000 4194304 4356188 140736612774608 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31279/statm: 81865 53971 162 40 0 80409 0
[pid=31279/tid=31286] ppid=31274 vsize=327460 CPUtime=4874.06 cores=4-7
/proc/31279/task/31286/stat : 31286 (plingeling) R 31274 31274 30879 0 -1 4202560 153375 1252 0 0 487385 21 0 0 25 0 5 0 508976457 335319040 53971 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31279/tid=31287] ppid=31274 vsize=327460 CPUtime=4902.4 cores=4-7
/proc/31279/task/31287/stat : 31287 (plingeling) R 31274 31274 30879 0 -1 4202560 147835 1252 0 0 490218 22 0 0 25 0 5 0 508976457 335319040 53971 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331007 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31279/tid=31288] ppid=31274 vsize=327460 CPUtime=4902.4 cores=4-7
/proc/31279/task/31288/stat : 31288 (plingeling) R 31274 31274 30879 0 -1 4202560 148441 1252 0 0 490216 24 0 0 25 0 5 0 508976457 335319040 53971 33554432000 4194304 4356188 140736612774608 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31279/tid=31289] ppid=31274 vsize=327460 CPUtime=4902.4 cores=4-7
/proc/31279/task/31289/stat : 31289 (plingeling) R 31274 31274 30879 0 -1 4202560 153628 1252 0 0 490220 20 0 0 25 0 5 0 508976457 335319040 53971 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39190.3
Current children cumulated vsize (KiB) 1485252

[startup+4962.3 s]
/proc/loadavg: 8.02 8.05 8.00 9/180 31437
/proc/meminfo: memFree=30110984/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
[pid=31275] ppid=31274 vsize=225600 CPUtime=4962.42 cores=0
/proc/31275/stat : 31275 (cryptominisat) R 31274 31274 30879 0 -1 4202496 296118 0 0 0 496113 129 0 0 25 0 1 0 508976455 231014400 34722 33554432000 4194304 4587354 140737139711552 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31275/statm: 56400 34722 336 96 0 52962 0
[pid=31276] ppid=31274 vsize=59132 CPUtime=4962.4 cores=1
/proc/31276/stat : 31276 (clasp) R 31274 31274 30879 0 -1 4202496 12974 0 0 0 496238 2 0 0 25 0 1 0 508976455 60551168 10478 33554432000 4194304 4785740 140735201266704 18446744073709551615 4287045 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31276/statm: 14783 10478 341 145 0 11828 0
[pid=31277] ppid=31274 vsize=456648 CPUtime=4961.79 cores=2
/proc/31277/stat : 31277 (TNM) R 31274 31274 30879 0 -1 4202496 555 0 0 0 496177 2 0 0 25 0 1 0 508976455 467607552 452 33554432000 134512640 134534564 4291756704 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31277/statm: 114162 452 95 6 0 112828 0
[pid=31278] ppid=31274 vsize=696364 CPUtime=4962.42 cores=3
/proc/31278/stat : 31278 (march_hi) R 31274 31274 30879 0 -1 4202496 116502 0 0 0 496155 87 0 0 25 0 1 0 508976455 713076736 115574 33554432000 4194304 4331988 140733648135968 18446744073709551615 4210430 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31278/statm: 174091 115574 175 34 0 172533 0
[pid=31279] ppid=31274 vsize=332324 CPUtime=19821 cores=4-7
/proc/31279/stat : 31279 (plingeling) S 31274 31274 30879 0 -1 4202496 610228 1252 0 0 1982008 89 0 0 18 0 5 0 508976455 340299776 55187 33554432000 4194304 4356188 140736612774608 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31279/statm: 83081 55187 162 40 0 81625 0
[pid=31279/tid=31286] ppid=31274 vsize=332324 CPUtime=4933.73 cores=4-7
/proc/31279/task/31286/stat : 31286 (plingeling) R 31274 31274 30879 0 -1 4202560 155295 1252 0 0 493352 21 0 0 25 0 5 0 508976457 340299776 55187 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31279/tid=31287] ppid=31274 vsize=332324 CPUtime=4962.4 cores=4-7
/proc/31279/task/31287/stat : 31287 (plingeling) R 31274 31274 30879 0 -1 4202560 150075 1252 0 0 496218 22 0 0 25 0 5 0 508976457 340299776 55187 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331039 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31279/tid=31288] ppid=31274 vsize=332324 CPUtime=4962.4 cores=4-7
/proc/31279/task/31288/stat : 31288 (plingeling) R 31274 31274 30879 0 -1 4202560 149721 1252 0 0 496216 24 0 0 25 0 5 0 508976457 340299776 55187 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31279/tid=31289] ppid=31274 vsize=332324 CPUtime=4962.4 cores=4-7
/proc/31279/task/31289/stat : 31289 (plingeling) R 31274 31274 30879 0 -1 4202560 153948 1252 0 0 496220 20 0 0 25 0 5 0 508976457 340299776 55187 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39670
Current children cumulated vsize (KiB) 1788392



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.01 8.04 8.00 9/180 31438
/proc/meminfo: memFree=30136244/32951124 swapFree=67106388/67111528
[pid=31274] ppid=31272 vsize=18324 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (ppfolio) S 31272 31274 30879 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 508976455 18763776 389 33554432000 4194304 4287625 140733875576672 18446744073709551615 212039675407 0 0 4096 65536 4601280656993943551 0 0 17 4 0 0 0
/proc/31274/statm: 4581 389 341 23 0 91 0
[pid=31275] ppid=31274 vsize=132460 CPUtime=5000.12 cores=0
/proc/31275/stat : 31275 (cryptominisat) R 31274 31274 30879 0 -1 4202496 296923 0 0 0 499882 130 0 0 25 0 1 0 508976455 135639040 29538 33554432000 4194304 4587354 140737139711552 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31275/statm: 33115 29538 336 96 0 29677 0
[pid=31276] ppid=31274 vsize=59132 CPUtime=5000.11 cores=1
/proc/31276/stat : 31276 (clasp) R 31274 31274 30879 0 -1 4202496 12974 0 0 0 500009 2 0 0 25 0 1 0 508976455 60551168 10478 33554432000 4194304 4785740 140735201266704 18446744073709551615 4553380 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31276/statm: 14783 10478 341 145 0 11828 0
[pid=31277] ppid=31274 vsize=456648 CPUtime=4999.5 cores=2
/proc/31277/stat : 31277 (TNM) R 31274 31274 30879 0 -1 4202496 555 0 0 0 499948 2 0 0 25 0 1 0 508976455 467607552 452 33554432000 134512640 134534564 4291756704 18446744073709551615 134530712 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31277/statm: 114162 452 95 6 0 112828 0
[pid=31278] ppid=31274 vsize=696364 CPUtime=5000.13 cores=3
/proc/31278/stat : 31278 (march_hi) R 31274 31274 30879 0 -1 4202496 117319 0 0 0 499926 87 0 0 25 0 1 0 508976455 713076736 116391 33554432000 4194304 4331988 140733648135968 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31278/statm: 174091 116391 175 34 0 172533 0
[pid=31279] ppid=31274 vsize=323364 CPUtime=19971.6 cores=4-7
/proc/31279/stat : 31279 (plingeling) S 31274 31274 30879 0 -1 4202496 612468 1252 0 0 1997069 89 0 0 18 0 5 0 508976455 331124736 52947 33554432000 4194304 4356188 140736612774608 18446744073709551615 212051458021 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31279/statm: 80841 52947 162 40 0 79385 0
[pid=31279/tid=31286] ppid=31274 vsize=323364 CPUtime=4971.22 cores=4-7
/proc/31279/task/31286/stat : 31286 (plingeling) R 31274 31274 30879 0 -1 4202560 155295 1252 0 0 497101 21 0 0 25 0 5 0 508976457 331124736 52947 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31279/tid=31287] ppid=31274 vsize=323364 CPUtime=5000.1 cores=4-7
/proc/31279/task/31287/stat : 31287 (plingeling) R 31274 31274 30879 0 -1 4202560 150075 1252 0 0 499988 22 0 0 25 0 5 0 508976457 331124736 52947 33554432000 4194304 4356188 140736612774608 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31279/tid=31288] ppid=31274 vsize=323364 CPUtime=5000.1 cores=4-7
/proc/31279/task/31288/stat : 31288 (plingeling) R 31274 31274 30879 0 -1 4202560 150041 1252 0 0 499986 24 0 0 25 0 5 0 508976457 331124736 52947 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331340 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31279/tid=31289] ppid=31274 vsize=323364 CPUtime=5000.11 cores=4-7
/proc/31279/task/31289/stat : 31289 (plingeling) R 31274 31274 30879 0 -1 4202560 155868 1252 0 0 499991 20 0 0 25 0 5 0 508976457 331124736 52947 33554432000 4194304 4356188 140736612774608 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39971.4
Current children cumulated vsize (KiB) 1686292

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 31274 and gives
#  childrusage.ru_utime.tv_sec=14998
#  childrusage.ru_utime.tv_usec=409895
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=367792
# CPU time returned by wait4() is 14999.8
# while last known CPU time is 39971.4
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39971.4
CPU user time (s): 39968.3
CPU system time (s): 3.1
CPU usage (%): 799.426
Max. virtual memory (cumulated for all children) (KiB): 2474708

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14998.4
system time used= 1.36779
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 310987
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 882
involuntary context switches= 45642

runsolver used 8.13076 second user time and 19.1491 second system time

The end

Launcher Data 

Begin job on node146 at 2011-04-30 14:13:22
IDJOB=3222417
IDBENCH=83086
IDSOLVER=1638
FILE ID=node146/3222417-1304165602
RUNJOBID= node146-1304165590-30893
PBS_JOBID= 13170949
Free space on /tmp= 73348 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/medium/unif-k7-r89-v80-c7120-S1968028041-077.UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3222417-1304165602/watcher-3222417-1304165602 -o /tmp/evaluation-result-3222417-1304165602/solver-3222417-1304165602 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3222417-1304165602.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= b69cb091f87b56cc59075b29ac9c632c
RANDOM SEED=1134870637

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30972004 kB
Buffers:        428032 kB
Cached:        1114200 kB
SwapCached:        700 kB
Active:         133740 kB
Inactive:      1467040 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30972004 kB
SwapTotal:    67111528 kB
SwapFree:     67106388 kB
Dirty:           10612 kB
Writeback:          52 kB
AnonPages:       58288 kB
Mapped:          14456 kB
Slab:           311180 kB
PageTables:       4368 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   185388 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73340 MiB
End job on node146 at 2011-04-30 15:36:44