Trace number 3219123

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39972 5000.02

General information on the benchmark

Namerandom/large/
unif-k7-r85-v200-c17000-S181603592-089-UNKNOWN.cnf
MD5SUMaf7b69f0071c372d5cdc250bc7f688e1
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2380.61
Satisfiable
(Un)Satisfiability was proved
Number of variables200
Number of clauses17000
Sum of the clauses size119000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 517000

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3219123-1304146704/watcher-3219123-1304146704 -o /tmp/evaluation-result-3219123-1304146704/solver-3219123-1304146704 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3219123-1304146704.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.11 6.22 3.60 2/170 31404
/proc/meminfo: memFree=23407176/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0

[startup+0.856005 s]
/proc/loadavg: 8.11 6.22 3.60 2/170 31404
/proc/meminfo: memFree=23407176/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 20 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.900996 s]
/proc/loadavg: 8.11 6.22 3.60 2/170 31404
/proc/meminfo: memFree=23407176/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 17 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.00098 s]
/proc/loadavg: 8.11 6.22 3.60 2/170 31404
/proc/meminfo: memFree=23407176/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 17 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.10096 s]
/proc/loadavg: 8.11 6.22 3.60 2/170 31404
/proc/meminfo: memFree=23407176/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 17 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50089 s]
/proc/loadavg: 8.11 6.22 3.60 2/170 31404
/proc/meminfo: memFree=23407176/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+3.10061 s]
/proc/loadavg: 8.10 6.25 3.62 9/180 31420
/proc/meminfo: memFree=23358248/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
[pid=31405] ppid=31404 vsize=1675288 CPUtime=3.08 cores=0
/proc/31405/stat : 31405 (cryptominisat) R 31404 31404 31286 0 -1 4202496 9679 0 0 0 307 1 0 0 25 0 1 0 604901114 1715494912 5346 33554432000 4194304 4587354 140735262276848 18446744073709551615 4364806 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31405/statm: 418822 5346 337 96 0 415384 0
[pid=31406] ppid=31404 vsize=19944 CPUtime=3.09 cores=1
/proc/31406/stat : 31406 (clasp) R 31404 31404 31286 0 -1 4202496 2432 0 0 0 309 0 0 0 25 0 1 0 604901114 20422656 2289 33554432000 4194304 4785740 140736452768048 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31406/statm: 4986 2289 340 145 0 2031 0
[pid=31407] ppid=31404 vsize=458072 CPUtime=3.09 cores=2
/proc/31407/stat : 31407 (TNM) R 31404 31404 31286 0 -1 4202496 1001 0 0 0 309 0 0 0 25 0 1 0 604901114 469065728 898 33554432000 134512640 134534564 4291975152 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31407/statm: 114518 898 95 6 0 113184 0
[pid=31408] ppid=31404 vsize=11768 CPUtime=3.09 cores=3
/proc/31408/stat : 31408 (march_hi) R 31404 31404 31286 0 -1 4202496 3828 0 0 0 309 0 0 0 25 0 1 0 604901114 12050432 1223 33554432000 4194304 4331988 140737167704464 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31408/statm: 2942 1223 174 34 0 1384 0
[pid=31409] ppid=31404 vsize=192060 CPUtime=12.24 cores=4-7
/proc/31409/stat : 31409 (plingeling) S 31404 31404 31286 0 -1 4202496 5995 1250 0 0 1223 1 0 0 18 0 5 0 604901114 196669440 3707 33554432000 4194304 4356188 140735095126384 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/31409/statm: 48015 3707 162 40 0 46559 0
[pid=31409/tid=31416] ppid=31404 vsize=192060 CPUtime=3.05 cores=4-7
/proc/31409/task/31416/stat : 31416 (plingeling) R 31404 31404 31286 0 -1 4202560 754 1250 0 0 305 0 0 0 25 0 5 0 604901118 196669440 3707 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331326 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31409/tid=31417] ppid=31404 vsize=192060 CPUtime=3.05 cores=4-7
/proc/31409/task/31417/stat : 31417 (plingeling) R 31404 31404 31286 0 -1 4202560 815 1250 0 0 305 0 0 0 25 0 5 0 604901118 196669440 3707 33554432000 4194304 4356188 140735095126384 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31409/tid=31418] ppid=31404 vsize=192060 CPUtime=3.04 cores=4-7
/proc/31409/task/31418/stat : 31418 (plingeling) R 31404 31404 31286 0 -1 4202560 836 1250 0 0 304 0 0 0 25 0 5 0 604901118 196669440 3707 33554432000 4194304 4356188 140735095126384 18446744073709551615 4239542 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31409/tid=31419] ppid=31404 vsize=192060 CPUtime=3.05 cores=4-7
/proc/31409/task/31419/stat : 31419 (plingeling) R 31404 31404 31286 0 -1 4202560 1064 1250 0 0 305 0 0 0 25 0 5 0 604901118 196669440 3707 33554432000 4194304 4356188 140735095126384 18446744073709551615 4234212 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.59
Current children cumulated vsize (KiB) 2375456

[startup+6.30705 s]
/proc/loadavg: 8.10 6.25 3.62 9/180 31420
/proc/meminfo: memFree=23354384/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
[pid=31405] ppid=31404 vsize=7815564 CPUtime=6.3 cores=0
/proc/31405/stat : 31405 (cryptominisat) R 31404 31404 31286 0 -1 4202496 10797 0 0 0 628 2 0 0 25 0 1 0 604901114 8003137536 5527 33554432000 4194304 4587354 140735262276848 18446744073709551615 4238755 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31405/statm: 1953891 5527 337 96 0 1950453 0
[pid=31406] ppid=31404 vsize=21904 CPUtime=6.3 cores=1
/proc/31406/stat : 31406 (clasp) R 31404 31404 31286 0 -1 4202496 2890 0 0 0 630 0 0 0 25 0 1 0 604901114 22429696 2747 33554432000 4194304 4785740 140736452768048 18446744073709551615 4287190 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31406/statm: 5476 2747 340 145 0 2521 0
[pid=31407] ppid=31404 vsize=458072 CPUtime=6.29 cores=2
/proc/31407/stat : 31407 (TNM) R 31404 31404 31286 0 -1 4202496 1001 0 0 0 629 0 0 0 25 0 1 0 604901114 469065728 898 33554432000 134512640 134534564 4291975152 18446744073709551615 134528708 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31407/statm: 114518 898 95 6 0 113184 0
[pid=31408] ppid=31404 vsize=11768 CPUtime=6.29 cores=3
/proc/31408/stat : 31408 (march_hi) R 31404 31404 31286 0 -1 4202496 3828 0 0 0 629 0 0 0 25 0 1 0 604901114 12050432 1223 33554432000 4194304 4331988 140737167704464 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31408/statm: 2942 1223 174 34 0 1384 0
[pid=31409] ppid=31404 vsize=193612 CPUtime=25.05 cores=4-7
/proc/31409/stat : 31409 (plingeling) S 31404 31404 31286 0 -1 4202496 6811 1250 0 0 2504 1 0 0 18 0 5 0 604901114 198258688 4127 33554432000 4194304 4356188 140735095126384 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/31409/statm: 48403 4127 162 40 0 46947 0
[pid=31409/tid=31416] ppid=31404 vsize=193612 CPUtime=6.25 cores=4-7
/proc/31409/task/31416/stat : 31416 (plingeling) R 31404 31404 31286 0 -1 4202560 1176 1250 0 0 625 0 0 0 25 0 5 0 604901118 198258688 4127 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31409/tid=31417] ppid=31404 vsize=193612 CPUtime=6.26 cores=4-7
/proc/31409/task/31417/stat : 31417 (plingeling) R 31404 31404 31286 0 -1 4202560 968 1250 0 0 626 0 0 0 25 0 5 0 604901118 198258688 4127 33554432000 4194304 4356188 140735095126384 18446744073709551615 4281652 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31409/tid=31418] ppid=31404 vsize=193612 CPUtime=6.22 cores=4-7
/proc/31409/task/31418/stat : 31418 (plingeling) R 31404 31404 31286 0 -1 4202560 884 1250 0 0 622 0 0 0 25 0 5 0 604901118 198258688 4127 33554432000 4194304 4356188 140735095126384 18446744073709551615 4270872 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31409/tid=31419] ppid=31404 vsize=193612 CPUtime=6.26 cores=4-7
/proc/31409/task/31419/stat : 31419 (plingeling) R 31404 31404 31286 0 -1 4202560 1257 1250 0 0 626 0 0 0 25 0 5 0 604901118 198258688 4127 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.23
Current children cumulated vsize (KiB) 8519244

[startup+12.7009 s]
/proc/loadavg: 8.09 6.28 3.65 9/180 31420
/proc/meminfo: memFree=23347192/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
[pid=31405] ppid=31404 vsize=7818960 CPUtime=12.68 cores=0
/proc/31405/stat : 31405 (cryptominisat) R 31404 31404 31286 0 -1 4202496 14393 0 0 0 1266 2 0 0 25 0 1 0 604901114 8006615040 6833 33554432000 4194304 4587354 140735262276848 18446744073709551615 4231728 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31405/statm: 1954740 6833 337 96 0 1951302 0
[pid=31406] ppid=31404 vsize=24560 CPUtime=12.69 cores=1
/proc/31406/stat : 31406 (clasp) R 31404 31404 31286 0 -1 4202496 3516 0 0 0 1269 0 0 0 25 0 1 0 604901114 25149440 3373 33554432000 4194304 4785740 140736452768048 18446744073709551615 4287008 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31406/statm: 6140 3373 340 145 0 3185 0
[pid=31407] ppid=31404 vsize=458072 CPUtime=12.69 cores=2
/proc/31407/stat : 31407 (TNM) R 31404 31404 31286 0 -1 4202496 1001 0 0 0 1269 0 0 0 25 0 1 0 604901114 469065728 898 33554432000 134512640 134534564 4291975152 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31407/statm: 114518 898 95 6 0 113184 0
[pid=31408] ppid=31404 vsize=13304 CPUtime=12.68 cores=3
/proc/31408/stat : 31408 (march_hi) R 31404 31404 31286 0 -1 4202496 4135 0 0 0 1268 0 0 0 25 0 1 0 604901114 13623296 1530 33554432000 4194304 4331988 140737167704464 18446744073709551615 4210448 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31408/statm: 3326 1530 175 34 0 1768 0
[pid=31409] ppid=31404 vsize=194272 CPUtime=50.59 cores=4-7
/proc/31409/stat : 31409 (plingeling) S 31404 31404 31286 0 -1 4202496 7712 1250 0 0 5058 1 0 0 18 0 5 0 604901114 198934528 4292 33554432000 4194304 4356188 140735095126384 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/31409/statm: 48568 4292 162 40 0 47112 0
[pid=31409/tid=31416] ppid=31404 vsize=194272 CPUtime=12.65 cores=4-7
/proc/31409/task/31416/stat : 31416 (plingeling) R 31404 31404 31286 0 -1 4202560 1433 1250 0 0 1265 0 0 0 25 0 5 0 604901118 198934528 4292 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31409/tid=31417] ppid=31404 vsize=194272 CPUtime=12.65 cores=4-7
/proc/31409/task/31417/stat : 31417 (plingeling) R 31404 31404 31286 0 -1 4202560 1226 1250 0 0 1265 0 0 0 25 0 5 0 604901118 198934528 4292 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31409/tid=31418] ppid=31404 vsize=194272 CPUtime=12.59 cores=4-7
/proc/31409/task/31418/stat : 31418 (plingeling) R 31404 31404 31286 0 -1 4202560 1109 1250 0 0 1259 0 0 0 25 0 5 0 604901118 198934528 4292 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31409/tid=31419] ppid=31404 vsize=194272 CPUtime=12.65 cores=4-7
/proc/31409/task/31419/stat : 31419 (plingeling) R 31404 31404 31286 0 -1 4202560 1418 1250 0 0 1265 0 0 0 25 0 5 0 604901118 198934528 4292 33554432000 4194304 4356188 140735095126384 18446744073709551615 4279478 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 101.33
Current children cumulated vsize (KiB) 8527492

[startup+25.5007 s]
/proc/loadavg: 8.07 6.37 3.71 9/180 31420
/proc/meminfo: memFree=23334816/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
[pid=31405] ppid=31404 vsize=1682900 CPUtime=25.49 cores=0
/proc/31405/stat : 31405 (cryptominisat) R 31404 31404 31286 0 -1 4202496 20062 0 0 0 2545 4 0 0 25 0 1 0 604901114 1723289600 8021 33554432000 4194304 4587354 140735262276848 18446744073709551615 4238568 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31405/statm: 420725 8021 337 96 0 417287 0
[pid=31406] ppid=31404 vsize=28728 CPUtime=25.49 cores=1
/proc/31406/stat : 31406 (clasp) R 31404 31404 31286 0 -1 4202496 4506 0 0 0 2549 0 0 0 25 0 1 0 604901114 29417472 4363 33554432000 4194304 4785740 140736452768048 18446744073709551615 4286941 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31406/statm: 7182 4363 340 145 0 4227 0
[pid=31407] ppid=31404 vsize=458072 CPUtime=25.49 cores=2
/proc/31407/stat : 31407 (TNM) R 31404 31404 31286 0 -1 4202496 1001 0 0 0 2549 0 0 0 25 0 1 0 604901114 469065728 898 33554432000 134512640 134534564 4291975152 18446744073709551615 134530320 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31407/statm: 114518 898 95 6 0 113184 0
[pid=31408] ppid=31404 vsize=15220 CPUtime=25.48 cores=3
/proc/31408/stat : 31408 (march_hi) R 31404 31404 31286 0 -1 4202496 4510 0 0 0 2548 0 0 0 25 0 1 0 604901114 15585280 1905 33554432000 4194304 4331988 140737167704464 18446744073709551615 4316436 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31408/statm: 3805 1905 175 34 0 2247 0
[pid=31409] ppid=31404 vsize=196468 CPUtime=101.72 cores=4-7
/proc/31409/stat : 31409 (plingeling) S 31404 31404 31286 0 -1 4202496 9413 1250 0 0 10170 2 0 0 18 0 5 0 604901114 201183232 4841 33554432000 4194304 4356188 140735095126384 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/31409/statm: 49117 4841 162 40 0 47661 0
[pid=31409/tid=31416] ppid=31404 vsize=196468 CPUtime=25.44 cores=4-7
/proc/31409/task/31416/stat : 31416 (plingeling) R 31404 31404 31286 0 -1 4202560 1818 1250 0 0 2544 0 0 0 25 0 5 0 604901118 201183232 4841 33554432000 4194304 4356188 140735095126384 18446744073709551615 4274287 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31409/tid=31417] ppid=31404 vsize=196468 CPUtime=25.45 cores=4-7
/proc/31409/task/31417/stat : 31417 (plingeling) R 31404 31404 31286 0 -1 4202560 1643 1250 0 0 2545 0 0 0 25 0 5 0 604901118 201183232 4841 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331268 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31409/tid=31418] ppid=31404 vsize=196468 CPUtime=25.31 cores=4-7
/proc/31409/task/31418/stat : 31418 (plingeling) R 31404 31404 31286 0 -1 4202560 1526 1250 0 0 2531 0 0 0 25 0 5 0 604901118 201183232 4841 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331045 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31409/tid=31419] ppid=31404 vsize=196468 CPUtime=25.45 cores=4-7
/proc/31409/task/31419/stat : 31419 (plingeling) R 31404 31404 31286 0 -1 4202560 1900 1250 0 0 2545 0 0 0 25 0 5 0 604901118 201183232 4841 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331176 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 203.67
Current children cumulated vsize (KiB) 2399712

################
# More data... #
################

[pid=31407] ppid=31404 vsize=458072 CPUtime=4782.38 cores=2
/proc/31407/stat : 31407 (TNM) R 31404 31404 31286 0 -1 4202496 1001 0 0 0 478237 1 0 0 25 0 1 0 604901114 469065728 898 33554432000 134512640 134534564 4291975152 18446744073709551615 134528877 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31407/statm: 114518 898 95 6 0 113184 0
[pid=31408] ppid=31404 vsize=691452 CPUtime=4782.39 cores=3
/proc/31408/stat : 31408 (march_hi) R 31404 31404 31286 0 -1 4202496 141123 0 0 0 478134 105 0 0 25 0 1 0 604901114 708046848 138518 33554432000 4194304 4331988 140737167704464 18446744073709551615 4299158 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31408/statm: 172863 138518 175 34 0 171305 0
[pid=31409] ppid=31404 vsize=261928 CPUtime=19102.1 cores=4-7
/proc/31409/stat : 31409 (plingeling) S 31404 31404 31286 0 -1 4202496 482533 1250 0 0 1910135 74 0 0 18 0 5 0 604901114 268214272 21206 33554432000 4194304 4356188 140735095126384 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/31409/statm: 65482 21206 162 40 0 64026 0
[pid=31409/tid=31416] ppid=31404 vsize=261928 CPUtime=4782.19 cores=4-7
/proc/31409/task/31416/stat : 31416 (plingeling) R 31404 31404 31286 0 -1 4202560 127263 1250 0 0 478198 21 0 0 25 0 5 0 604901118 268214272 21206 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331151 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31409/tid=31417] ppid=31404 vsize=261928 CPUtime=4781.88 cores=4-7
/proc/31409/task/31417/stat : 31417 (plingeling) R 31404 31404 31286 0 -1 4202560 117392 1250 0 0 478171 17 0 0 25 0 5 0 604901118 268214272 21206 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31409/tid=31418] ppid=31404 vsize=261928 CPUtime=4756.26 cores=4-7
/proc/31409/task/31418/stat : 31418 (plingeling) R 31404 31404 31286 0 -1 4202560 113275 1250 0 0 475611 15 0 0 25 0 5 0 604901118 268214272 21206 33554432000 4194304 4356188 140735095126384 18446744073709551615 4279435 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31409/tid=31419] ppid=31404 vsize=261928 CPUtime=4781.69 cores=4-7
/proc/31409/task/31419/stat : 31419 (plingeling) R 31404 31404 31286 0 -1 4202560 122077 1250 0 0 478150 19 0 0 25 0 5 0 604901118 268214272 21206 33554432000 4194304 4356188 140735095126384 18446744073709551615 4330973 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38231.5
Current children cumulated vsize (KiB) 3411896

[startup+4842.3 s]
/proc/loadavg: 8.05 8.03 8.00 10/180 31569
/proc/meminfo: memFree=22519004/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
[pid=31405] ppid=31404 vsize=100588 CPUtime=4842.39 cores=0
/proc/31405/stat : 31405 (cryptominisat) R 31404 31404 31286 0 -1 4202496 823527 0 0 0 483892 347 0 0 25 0 1 0 604901114 103002112 18984 33554432000 4194304 4587354 140735262276848 18446744073709551615 4238568 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31405/statm: 25147 18984 337 96 0 21709 0
[pid=31406] ppid=31404 vsize=124720 CPUtime=4842.25 cores=1
/proc/31406/stat : 31406 (clasp) R 31404 31404 31286 0 -1 4202496 27071 0 0 0 484220 5 0 0 25 0 1 0 604901114 127713280 25455 33554432000 4194304 4785740 140736452768048 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31406/statm: 31180 25455 341 145 0 28225 0
[pid=31407] ppid=31404 vsize=458072 CPUtime=4842.39 cores=2
/proc/31407/stat : 31407 (TNM) R 31404 31404 31286 0 -1 4202496 1001 0 0 0 484238 1 0 0 25 0 1 0 604901114 469065728 898 33554432000 134512640 134534564 4291975152 18446744073709551615 134528523 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31407/statm: 114518 898 95 6 0 113184 0
[pid=31408] ppid=31404 vsize=691452 CPUtime=4842.39 cores=3
/proc/31408/stat : 31408 (march_hi) R 31404 31404 31286 0 -1 4202496 142863 0 0 0 484132 107 0 0 25 0 1 0 604901114 708046848 140258 33554432000 4194304 4331988 140737167704464 18446744073709551615 4239602 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31408/statm: 172863 140258 175 34 0 171305 0
[pid=31409] ppid=31404 vsize=293928 CPUtime=19341.8 cores=4-7
/proc/31409/stat : 31409 (plingeling) S 31404 31404 31286 0 -1 4202496 490533 1250 0 0 1934101 75 0 0 18 0 5 0 604901114 300982272 29206 33554432000 4194304 4356188 140735095126384 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/31409/statm: 73482 29206 162 40 0 72026 0
[pid=31409/tid=31416] ppid=31404 vsize=293928 CPUtime=4842.18 cores=4-7
/proc/31409/task/31416/stat : 31416 (plingeling) R 31404 31404 31286 0 -1 4202560 129535 1250 0 0 484197 21 0 0 25 0 5 0 604901118 300982272 29206 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31409/tid=31417] ppid=31404 vsize=293928 CPUtime=4841.88 cores=4-7
/proc/31409/task/31417/stat : 31417 (plingeling) R 31404 31404 31286 0 -1 4202560 120848 1250 0 0 484171 17 0 0 25 0 5 0 604901118 300982272 29206 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31409/tid=31418] ppid=31404 vsize=293928 CPUtime=4815.93 cores=4-7
/proc/31409/task/31418/stat : 31418 (plingeling) R 31404 31404 31286 0 -1 4202560 113275 1250 0 0 481578 15 0 0 25 0 5 0 604901118 300982272 29206 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31409/tid=31419] ppid=31404 vsize=293928 CPUtime=4841.69 cores=4-7
/proc/31409/task/31419/stat : 31419 (plingeling) R 31404 31404 31286 0 -1 4202560 124349 1250 0 0 484150 19 0 0 25 0 5 0 604901118 300982272 29206 33554432000 4194304 4356188 140735095126384 18446744073709551615 4279535 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38711.2
Current children cumulated vsize (KiB) 1687084

[startup+4902.3 s]
/proc/loadavg: 8.05 8.03 8.00 9/180 31571
/proc/meminfo: memFree=22514712/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
[pid=31405] ppid=31404 vsize=180568 CPUtime=4902.39 cores=0
/proc/31405/stat : 31405 (cryptominisat) R 31404 31404 31286 0 -1 4202496 829713 0 0 0 489888 351 0 0 25 0 1 0 604901114 184901632 25064 33554432000 4194304 4587354 140735262276848 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31405/statm: 45142 25064 337 96 0 41704 0
[pid=31406] ppid=31404 vsize=125284 CPUtime=4902.26 cores=1
/proc/31406/stat : 31406 (clasp) R 31404 31404 31286 0 -1 4202496 27335 0 0 0 490221 5 0 0 25 0 1 0 604901114 128290816 25575 33554432000 4194304 4785740 140736452768048 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31406/statm: 31321 25575 341 145 0 28366 0
[pid=31407] ppid=31404 vsize=458072 CPUtime=4902.39 cores=2
/proc/31407/stat : 31407 (TNM) R 31404 31404 31286 0 -1 4202496 1001 0 0 0 490238 1 0 0 25 0 1 0 604901114 469065728 898 33554432000 134512640 134534564 4291975152 18446744073709551615 134528483 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31407/statm: 114518 898 95 6 0 113184 0
[pid=31408] ppid=31404 vsize=691452 CPUtime=4902.4 cores=3
/proc/31408/stat : 31408 (march_hi) R 31404 31404 31286 0 -1 4202496 144588 0 0 0 490131 109 0 0 25 0 1 0 604901114 708046848 141983 33554432000 4194304 4331988 140737167704464 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31408/statm: 172863 141983 175 34 0 171305 0
[pid=31409] ppid=31404 vsize=298664 CPUtime=19581.4 cores=4-7
/proc/31409/stat : 31409 (plingeling) S 31404 31404 31286 0 -1 4202496 497029 1250 0 0 1958069 76 0 0 18 0 5 0 604901114 305831936 30390 33554432000 4194304 4356188 140735095126384 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/31409/statm: 74666 30390 162 40 0 73210 0
[pid=31409/tid=31416] ppid=31404 vsize=298664 CPUtime=4902.19 cores=4-7
/proc/31409/task/31416/stat : 31416 (plingeling) R 31404 31404 31286 0 -1 4202560 130943 1250 0 0 490198 21 0 0 25 0 5 0 604901118 305831936 30390 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31409/tid=31417] ppid=31404 vsize=298664 CPUtime=4901.87 cores=4-7
/proc/31409/task/31417/stat : 31417 (plingeling) R 31404 31404 31286 0 -1 4202560 120848 1250 0 0 490170 17 0 0 25 0 5 0 604901118 305831936 30390 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331178 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31409/tid=31418] ppid=31404 vsize=298664 CPUtime=4875.62 cores=4-7
/proc/31409/task/31418/stat : 31418 (plingeling) R 31404 31404 31286 0 -1 4202560 116955 1250 0 0 487546 16 0 0 25 0 5 0 604901118 305831936 30390 33554432000 4194304 4356188 140735095126384 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31409/tid=31419] ppid=31404 vsize=298664 CPUtime=4901.7 cores=4-7
/proc/31409/task/31419/stat : 31419 (plingeling) R 31404 31404 31286 0 -1 4202560 125757 1250 0 0 490151 19 0 0 25 0 5 0 604901118 305831936 30390 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39190.9
Current children cumulated vsize (KiB) 1772364

[startup+4962.3 s]
/proc/loadavg: 8.02 8.02 8.00 9/180 31573
/proc/meminfo: memFree=22546392/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
[pid=31405] ppid=31404 vsize=1857976 CPUtime=4962.39 cores=0
/proc/31405/stat : 31405 (cryptominisat) R 31404 31404 31286 0 -1 4202496 837434 0 0 0 495885 354 0 0 25 0 1 0 604901114 1902567424 22155 33554432000 4194304 4587354 140735262276848 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31405/statm: 464494 22155 337 96 0 461056 0
[pid=31406] ppid=31404 vsize=127136 CPUtime=4962.25 cores=1
/proc/31406/stat : 31406 (clasp) R 31404 31404 31286 0 -1 4202496 27813 0 0 0 496220 5 0 0 25 0 1 0 604901114 130187264 25981 33554432000 4194304 4785740 140736452768048 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31406/statm: 31784 25981 341 145 0 28829 0
[pid=31407] ppid=31404 vsize=458072 CPUtime=4962.39 cores=2
/proc/31407/stat : 31407 (TNM) R 31404 31404 31286 0 -1 4202496 1001 0 0 0 496238 1 0 0 25 0 1 0 604901114 469065728 898 33554432000 134512640 134534564 4291975152 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31407/statm: 114518 898 95 6 0 113184 0
[pid=31408] ppid=31404 vsize=691452 CPUtime=4962.39 cores=3
/proc/31408/stat : 31408 (march_hi) R 31404 31404 31286 0 -1 4202496 146246 0 0 0 496129 110 0 0 25 0 1 0 604901114 708046848 143641 33554432000 4194304 4331988 140737167704464 18446744073709551615 4210516 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31408/statm: 172863 143641 175 34 0 171305 0
[pid=31409] ppid=31404 vsize=270120 CPUtime=19821.1 cores=4-7
/proc/31409/stat : 31409 (plingeling) S 31404 31404 31286 0 -1 4202496 499301 1250 0 0 1982034 76 0 0 18 0 5 0 604901114 276602880 23254 33554432000 4194304 4356188 140735095126384 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/31409/statm: 67530 23254 162 40 0 66074 0
[pid=31409/tid=31416] ppid=31404 vsize=270120 CPUtime=4962.18 cores=4-7
/proc/31409/task/31416/stat : 31416 (plingeling) R 31404 31404 31286 0 -1 4202560 130943 1250 0 0 496197 21 0 0 25 0 5 0 604901118 276602880 23254 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31409/tid=31417] ppid=31404 vsize=270120 CPUtime=4961.87 cores=4-7
/proc/31409/task/31417/stat : 31417 (plingeling) R 31404 31404 31286 0 -1 4202560 123120 1250 0 0 496169 18 0 0 25 0 5 0 604901118 276602880 23254 33554432000 4194304 4356188 140735095126384 18446744073709551615 4332260 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31409/tid=31418] ppid=31404 vsize=270120 CPUtime=4935.29 cores=4-7
/proc/31409/task/31418/stat : 31418 (plingeling) R 31404 31404 31286 0 -1 4202560 116955 1250 0 0 493513 16 0 0 25 0 5 0 604901118 276602880 23254 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331007 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31409/tid=31419] ppid=31404 vsize=270120 CPUtime=4961.69 cores=4-7
/proc/31409/task/31419/stat : 31419 (plingeling) R 31404 31404 31286 0 -1 4202560 125757 1250 0 0 496150 19 0 0 25 0 5 0 604901118 276602880 23254 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39670.5
Current children cumulated vsize (KiB) 3423080



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.01 8.02 8.00 9/180 31574
/proc/meminfo: memFree=22514500/32951124 swapFree=67102140/67111528
[pid=31404] ppid=31402 vsize=18324 CPUtime=0 cores=0-7
/proc/31404/stat : 31404 (ppfolio) S 31402 31404 31286 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 604901114 18763776 389 33554432000 4194304 4287625 140733560388976 18446744073709551615 216735198735 0 0 4096 65536 4522079810788261887 0 0 17 4 0 0 0
/proc/31404/statm: 4581 389 341 23 0 91 0
[pid=31405] ppid=31404 vsize=1857976 CPUtime=5000.1 cores=0
/proc/31405/stat : 31405 (cryptominisat) R 31404 31404 31286 0 -1 4202496 840674 0 0 0 499654 356 0 0 25 0 1 0 604901114 1902567424 25339 33554432000 4194304 4587354 140735262276848 18446744073709551615 4238618 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/31405/statm: 464494 25339 337 96 0 461056 0
[pid=31406] ppid=31404 vsize=127704 CPUtime=4999.96 cores=1
/proc/31406/stat : 31406 (clasp) R 31404 31404 31286 0 -1 4202496 28016 0 0 0 499991 5 0 0 25 0 1 0 604901114 130768896 26112 33554432000 4194304 4785740 140736452768048 18446744073709551615 4553411 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/31406/statm: 31926 26112 341 145 0 28971 0
[pid=31407] ppid=31404 vsize=458072 CPUtime=5000.1 cores=2
/proc/31407/stat : 31407 (TNM) R 31404 31404 31286 0 -1 4202496 1001 0 0 0 500009 1 0 0 25 0 1 0 604901114 469065728 898 33554432000 134512640 134534564 4291975152 18446744073709551615 134529765 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/31407/statm: 114518 898 95 6 0 113184 0
[pid=31408] ppid=31404 vsize=691452 CPUtime=5000.11 cores=3
/proc/31408/stat : 31408 (march_hi) R 31404 31404 31286 0 -1 4202496 147316 0 0 0 499900 111 0 0 25 0 1 0 604901114 708046848 144711 33554432000 4194304 4331988 140737167704464 18446744073709551615 4246241 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/31408/statm: 172863 144711 175 34 0 171305 0
[pid=31409] ppid=31404 vsize=284840 CPUtime=19971.7 cores=4-7
/proc/31409/stat : 31409 (plingeling) S 31404 31404 31286 0 -1 4202496 502981 1250 0 0 1997095 76 0 0 18 0 5 0 604901114 291676160 26934 33554432000 4194304 4356188 140735095126384 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/31409/statm: 71210 26934 162 40 0 69754 0
[pid=31409/tid=31416] ppid=31404 vsize=284840 CPUtime=4999.89 cores=4-7
/proc/31409/task/31416/stat : 31416 (plingeling) R 31404 31404 31286 0 -1 4202560 133215 1250 0 0 499968 21 0 0 25 0 5 0 604901118 291676160 26934 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=31409/tid=31417] ppid=31404 vsize=284840 CPUtime=4999.57 cores=4-7
/proc/31409/task/31417/stat : 31417 (plingeling) R 31404 31404 31286 0 -1 4202560 124528 1250 0 0 499939 18 0 0 25 0 5 0 604901118 291676160 26934 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31409/tid=31418] ppid=31404 vsize=284840 CPUtime=4972.78 cores=4-7
/proc/31409/task/31418/stat : 31418 (plingeling) R 31404 31404 31286 0 -1 4202560 116955 1250 0 0 497262 16 0 0 25 0 5 0 604901118 291676160 26934 33554432000 4194304 4356188 140735095126384 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31409/tid=31419] ppid=31404 vsize=284840 CPUtime=4999.4 cores=4-7
/proc/31409/task/31419/stat : 31419 (plingeling) R 31404 31404 31286 0 -1 4202560 125757 1250 0 0 499921 19 0 0 25 0 5 0 604901118 291676160 26934 33554432000 4194304 4356188 140735095126384 18446744073709551615 4331119 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39972
Current children cumulated vsize (KiB) 3438368

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 31404 and gives
#  childrusage.ru_utime.tv_sec=14996
#  childrusage.ru_utime.tv_usec=546178
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=654444
# CPU time returned by wait4() is 15000.2
# while last known CPU time is 39972
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39972
CPU user time (s): 39966.5
CPU system time (s): 5.49
CPU usage (%): 799.437
Max. virtual memory (cumulated for all children) (KiB): 17018556

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14996.5
system time used= 3.65444
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 870226
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 667
involuntary context switches= 80899

runsolver used 7.49886 second user time and 19.776 second system time

The end

Launcher Data 

Begin job on node119 at 2011-04-30 08:58:25
IDJOB=3219123
IDBENCH=82537
IDSOLVER=1638
FILE ID=node119/3219123-1304146704
RUNJOBID= node119-1304146306-31300
PBS_JOBID= 13170899
Free space on /tmp= 73016 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k7-r85-v200-c17000-S181603592-089-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3219123-1304146704/watcher-3219123-1304146704 -o /tmp/evaluation-result-3219123-1304146704/solver-3219123-1304146704 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3219123-1304146704.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= af7b69f0071c372d5cdc250bc7f688e1
RANDOM SEED=1894991545

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23407580 kB
Buffers:       1945828 kB
Cached:        6961532 kB
SwapCached:       3688 kB
Active:         433376 kB
Inactive:      8532944 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23407580 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:           10484 kB
Writeback:           0 kB
AnonPages:       59052 kB
Mapped:          14616 kB
Slab:           512040 kB
PageTables:       4068 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   184624 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73008 MiB
End job on node119 at 2011-04-30 10:21:47