Trace number 3275081

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 40000.4 5003.62

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v2640c305320gyes5.cnf
MD5SUMf892a2bda6fe6aec6f2fcb4e1c541df8
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2640
Number of clauses305320
Sum of the clauses size613120
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2305240
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 580

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3275081-1304145216/watcher-3275081-1304145216 -o /tmp/evaluation-result-3275081-1304145216/solver-3275081-1304145216 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3275081-1304145216.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.92 1.63 2.75 1/170 11412
/proc/meminfo: memFree=23004716/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) R 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0

[startup+0.067356 s]
/proc/loadavg: 0.92 1.63 2.75 1/170 11412
/proc/meminfo: memFree=23004716/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 6 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100353 s]
/proc/loadavg: 0.92 1.63 2.75 1/170 11412
/proc/meminfo: memFree=23004716/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) R 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300322 s]
/proc/loadavg: 0.92 1.63 2.75 1/170 11412
/proc/meminfo: memFree=23004716/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 24 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 4 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700251 s]
/proc/loadavg: 0.92 1.63 2.75 1/170 11412
/proc/meminfo: memFree=23004716/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 24 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 4 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50111 s]
/proc/loadavg: 0.92 1.63 2.75 9/180 11427
/proc/meminfo: memFree=22842188/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 4 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
[pid=11413] ppid=11412 vsize=80860 CPUtime=1.49 cores=0
/proc/11413/stat : 11413 (cryptominisat) R 11412 11412 11386 0 -1 4202496 17382 0 0 0 146 3 0 0 25 0 1 0 526121265 82800640 13940 33554432000 4194304 4587354 140734921408064 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11413/statm: 20215 13940 326 96 0 16777 0
[pid=11414] ppid=11412 vsize=33932 CPUtime=1.49 cores=1
/proc/11414/stat : 11414 (clasp) R 11412 11412 11386 0 -1 4202496 7077 0 0 0 148 1 0 0 20 0 1 0 526121265 34746368 5794 33554432000 4194304 4785740 140736627916064 18446744073709551615 4482821 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11414/statm: 8483 5794 317 145 0 5528 0
[pid=11415] ppid=11412 vsize=466252 CPUtime=1.48 cores=2
/proc/11415/stat : 11415 (TNM) R 11412 11412 11386 0 -1 4202496 5032 0 0 0 148 0 0 0 20 0 1 0 526121265 477442048 4929 33554432000 134512640 134534564 4287706240 18446744073709551615 134524736 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11415/statm: 116563 4929 91 6 0 115229 0
[pid=11416] ppid=11412 vsize=47888 CPUtime=1.49 cores=3
/proc/11416/stat : 11416 (march_hi) R 11412 11412 11386 0 -1 4202496 23180 0 0 0 146 3 0 0 20 0 1 0 526121265 49037312 5330 33554432000 4194304 4331988 140733887521152 18446744073709551615 4210400 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11416/statm: 11972 5330 172 34 0 10414 0
[pid=11417] ppid=11412 vsize=98792 CPUtime=5.36 cores=4-7
/proc/11417/stat : 11417 (plingeling) S 11412 11412 11386 0 -1 4202496 17422 1251 0 0 534 2 0 0 20 0 5 0 526121265 101163008 13123 33554432000 4194304 4356188 140735639286288 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11417/statm: 24698 13123 163 40 0 23242 0
[pid=11417/tid=11424] ppid=11412 vsize=98792 CPUtime=1.28 cores=4-7
/proc/11417/task/11424/stat : 11424 (plingeling) R 11412 11412 11386 0 -1 4202560 1855 1251 0 0 128 0 0 0 19 0 5 0 526121285 101163008 13123 33554432000 4194304 4356188 140735639286288 18446744073709551615 4211822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11417/tid=11425] ppid=11412 vsize=98792 CPUtime=1.28 cores=4-7
/proc/11417/task/11425/stat : 11425 (plingeling) R 11412 11412 11386 0 -1 4202560 2557 1251 0 0 128 0 0 0 25 0 5 0 526121285 101163008 13123 33554432000 4194304 4356188 140735639286288 18446744073709551615 4271040 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11417/tid=11426] ppid=11412 vsize=98792 CPUtime=1.28 cores=4-7
/proc/11417/task/11426/stat : 11426 (plingeling) R 11412 11412 11386 0 -1 4202560 2492 1251 0 0 128 0 0 0 25 0 5 0 526121285 101163008 13123 33554432000 4194304 4356188 140735639286288 18446744073709551615 4249586 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11417/tid=11427] ppid=11412 vsize=98792 CPUtime=1.28 cores=4-7
/proc/11417/task/11427/stat : 11427 (plingeling) R 11412 11412 11386 0 -1 4202560 1734 1251 0 0 128 0 0 0 25 0 5 0 526121285 101163008 13123 33554432000 4194304 4356188 140735639286288 18446744073709551615 4281722 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.31
Current children cumulated vsize (KiB) 746048

[startup+3.10083 s]
/proc/loadavg: 1.49 1.74 2.77 9/180 11427
/proc/meminfo: memFree=22822100/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 4 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
[pid=11413] ppid=11412 vsize=1867496 CPUtime=3.08 cores=0
/proc/11413/stat : 11413 (cryptominisat) R 11412 11412 11386 0 -1 4202496 19497 0 0 0 305 3 0 0 25 0 1 0 526121265 1912315904 14150 33554432000 4194304 4587354 140734921408064 18446744073709551615 4420280 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11413/statm: 466874 14150 331 96 0 463436 0
[pid=11414] ppid=11412 vsize=32376 CPUtime=3.09 cores=1
/proc/11414/stat : 11414 (clasp) R 11412 11412 11386 0 -1 4202496 7306 0 0 0 308 1 0 0 25 0 1 0 526121265 33153024 5426 33554432000 4194304 4785740 140736627916064 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11414/statm: 8094 5426 341 145 0 5139 0
[pid=11415] ppid=11412 vsize=472460 CPUtime=3.08 cores=2
/proc/11415/stat : 11415 (TNM) R 11412 11412 11386 0 -1 4202496 7210 0 0 0 308 0 0 0 25 0 1 0 526121265 483799040 7107 33554432000 134512640 134534564 4287706240 18446744073709551615 134528988 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11415/statm: 118115 7107 95 6 0 116781 0
[pid=11416] ppid=11412 vsize=48688 CPUtime=3.09 cores=3
/proc/11416/stat : 11416 (march_hi) R 11412 11412 11386 0 -1 4202496 23386 0 0 0 306 3 0 0 25 0 1 0 526121265 49856512 5536 33554432000 4194304 4331988 140733887521152 18446744073709551615 4228235 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11416/statm: 12172 5536 175 34 0 10614 0
[pid=11417] ppid=11412 vsize=111336 CPUtime=11.74 cores=4-7
/proc/11417/stat : 11417 (plingeling) S 11412 11412 11386 0 -1 4202496 20558 1251 0 0 1172 2 0 0 20 0 5 0 526121265 114008064 16259 33554432000 4194304 4356188 140735639286288 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11417/statm: 27834 16259 163 40 0 26378 0
[pid=11417/tid=11424] ppid=11412 vsize=111336 CPUtime=2.88 cores=4-7
/proc/11417/task/11424/stat : 11424 (plingeling) R 11412 11412 11386 0 -1 4202560 2110 1251 0 0 288 0 0 0 23 0 5 0 526121285 114008064 16259 33554432000 4194304 4356188 140735639286288 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11417/tid=11425] ppid=11412 vsize=111336 CPUtime=2.88 cores=4-7
/proc/11417/task/11425/stat : 11425 (plingeling) R 11412 11412 11386 0 -1 4202560 2941 1251 0 0 288 0 0 0 25 0 5 0 526121285 114008064 16259 33554432000 4194304 4356188 140735639286288 18446744073709551615 4330805 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11417/tid=11426] ppid=11412 vsize=111336 CPUtime=2.87 cores=4-7
/proc/11417/task/11426/stat : 11426 (plingeling) R 11412 11412 11386 0 -1 4202560 3388 1251 0 0 287 0 0 0 25 0 5 0 526121285 114008064 16259 33554432000 4194304 4356188 140735639286288 18446744073709551615 4335971 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11417/tid=11427] ppid=11412 vsize=111336 CPUtime=2.88 cores=4-7
/proc/11417/task/11427/stat : 11427 (plingeling) R 11412 11412 11386 0 -1 4202560 3335 1251 0 0 288 0 0 0 25 0 5 0 526121285 114008064 16259 33554432000 4194304 4356188 140735639286288 18446744073709551615 4281755 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.08
Current children cumulated vsize (KiB) 2550680

[startup+6.30126 s]
/proc/loadavg: 1.49 1.74 2.77 9/180 11427
/proc/meminfo: memFree=22804012/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 4 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
[pid=11413] ppid=11412 vsize=1867496 CPUtime=6.28 cores=0
/proc/11413/stat : 11413 (cryptominisat) R 11412 11412 11386 0 -1 4202496 19907 0 0 0 625 3 0 0 25 0 1 0 526121265 1912315904 14553 33554432000 4194304 4587354 140734921408064 18446744073709551615 4365216 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11413/statm: 466874 14553 337 96 0 463436 0
[pid=11414] ppid=11412 vsize=38672 CPUtime=6.29 cores=1
/proc/11414/stat : 11414 (clasp) R 11412 11412 11386 0 -1 4202496 8877 0 0 0 627 2 0 0 25 0 1 0 526121265 39600128 6997 33554432000 4194304 4785740 140736627916064 18446744073709551615 274473628357 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11414/statm: 9668 6997 341 145 0 6713 0
[pid=11415] ppid=11412 vsize=472460 CPUtime=6.28 cores=2
/proc/11415/stat : 11415 (TNM) R 11412 11412 11386 0 -1 4202496 7210 0 0 0 628 0 0 0 25 0 1 0 526121265 483799040 7107 33554432000 134512640 134534564 4287706240 18446744073709551615 134528730 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11415/statm: 118115 7107 95 6 0 116781 0
[pid=11416] ppid=11412 vsize=49100 CPUtime=6.29 cores=3
/proc/11416/stat : 11416 (march_hi) R 11412 11412 11386 0 -1 4202496 23465 0 0 0 626 3 0 0 25 0 1 0 526121265 50278400 5614 33554432000 4194304 4331988 140733887521152 18446744073709551615 4226237 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11416/statm: 12275 5614 175 34 0 10717 0
[pid=11417] ppid=11412 vsize=124028 CPUtime=24.53 cores=4-7
/proc/11417/stat : 11417 (plingeling) S 11412 11412 11386 0 -1 4202496 23731 1251 0 0 2450 3 0 0 20 0 5 0 526121265 127004672 19432 33554432000 4194304 4356188 140735639286288 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11417/statm: 31007 19432 163 40 0 29551 0
[pid=11417/tid=11424] ppid=11412 vsize=124028 CPUtime=6.08 cores=4-7
/proc/11417/task/11424/stat : 11424 (plingeling) R 11412 11412 11386 0 -1 4202560 2896 1251 0 0 608 0 0 0 25 0 5 0 526121285 127004672 19432 33554432000 4194304 4356188 140735639286288 18446744073709551615 4270866 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11417/tid=11425] ppid=11412 vsize=124028 CPUtime=6.08 cores=4-7
/proc/11417/task/11425/stat : 11425 (plingeling) R 11412 11412 11386 0 -1 4202560 3902 1251 0 0 608 0 0 0 25 0 5 0 526121285 127004672 19432 33554432000 4194304 4356188 140735639286288 18446744073709551615 4210544 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11417/tid=11426] ppid=11412 vsize=124028 CPUtime=6.05 cores=4-7
/proc/11417/task/11426/stat : 11426 (plingeling) R 11412 11412 11386 0 -1 4202560 4303 1251 0 0 605 0 0 0 25 0 5 0 526121285 127004672 19432 33554432000 4194304 4356188 140735639286288 18446744073709551615 4270615 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11417/tid=11427] ppid=11412 vsize=124028 CPUtime=6.08 cores=4-7
/proc/11417/task/11427/stat : 11427 (plingeling) R 11412 11412 11386 0 -1 4202560 3846 1251 0 0 608 0 0 0 25 0 5 0 526121285 127004672 19432 33554432000 4194304 4356188 140735639286288 18446744073709551615 4332109 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 49.67
Current children cumulated vsize (KiB) 2570080

[startup+12.7011 s]
/proc/loadavg: 2.49 1.94 2.83 9/180 11427
/proc/meminfo: memFree=22758864/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 4 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
[pid=11413] ppid=11412 vsize=1892392 CPUtime=12.69 cores=0
/proc/11413/stat : 11413 (cryptominisat) R 11412 11412 11386 0 -1 4202496 25846 0 0 0 1265 4 0 0 25 0 1 0 526121265 1937809408 20490 33554432000 4194304 4587354 140734921408064 18446744073709551615 4388336 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11413/statm: 473098 20490 337 96 0 469660 0
[pid=11414] ppid=11412 vsize=56848 CPUtime=12.68 cores=1
/proc/11414/stat : 11414 (clasp) R 11412 11412 11386 0 -1 4202496 13421 0 0 0 1265 3 0 0 25 0 1 0 526121265 58212352 11541 33554432000 4194304 4785740 140736627916064 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11414/statm: 14212 11541 341 145 0 11257 0
[pid=11415] ppid=11412 vsize=472460 CPUtime=12.68 cores=2
/proc/11415/stat : 11415 (TNM) R 11412 11412 11386 0 -1 4202496 7210 0 0 0 1268 0 0 0 25 0 1 0 526121265 483799040 7107 33554432000 134512640 134534564 4287706240 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11415/statm: 118115 7107 95 6 0 116781 0
[pid=11416] ppid=11412 vsize=49580 CPUtime=12.69 cores=3
/proc/11416/stat : 11416 (march_hi) R 11412 11412 11386 0 -1 4202496 23598 0 0 0 1266 3 0 0 25 0 1 0 526121265 50769920 5746 33554432000 4194304 4331988 140733887521152 18446744073709551615 4226263 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11416/statm: 12395 5746 175 34 0 10837 0
[pid=11417] ppid=11412 vsize=130684 CPUtime=50.1 cores=4-7
/proc/11417/stat : 11417 (plingeling) S 11412 11412 11386 0 -1 4202496 25395 1251 0 0 5007 3 0 0 20 0 5 0 526121265 133820416 21096 33554432000 4194304 4356188 140735639286288 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11417/statm: 32671 21096 163 40 0 31215 0
[pid=11417/tid=11424] ppid=11412 vsize=130684 CPUtime=12.48 cores=4-7
/proc/11417/task/11424/stat : 11424 (plingeling) R 11412 11412 11386 0 -1 4202560 2896 1251 0 0 1248 0 0 0 25 0 5 0 526121285 133820416 21096 33554432000 4194304 4356188 140735639286288 18446744073709551615 4330973 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11417/tid=11425] ppid=11412 vsize=130684 CPUtime=12.48 cores=4-7
/proc/11417/task/11425/stat : 11425 (plingeling) R 11412 11412 11386 0 -1 4202560 4158 1251 0 0 1248 0 0 0 25 0 5 0 526121285 133820416 21096 33554432000 4194304 4356188 140735639286288 18446744073709551615 4284008 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11417/tid=11426] ppid=11412 vsize=130684 CPUtime=12.42 cores=4-7
/proc/11417/task/11426/stat : 11426 (plingeling) R 11412 11412 11386 0 -1 4202560 4431 1251 0 0 1242 0 0 0 25 0 5 0 526121285 133820416 21096 33554432000 4194304 4356188 140735639286288 18446744073709551615 4212048 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11417/tid=11427] ppid=11412 vsize=130684 CPUtime=12.48 cores=4-7
/proc/11417/task/11427/stat : 11427 (plingeling) R 11412 11412 11386 0 -1 4202560 5126 1251 0 0 1248 0 0 0 25 0 5 0 526121285 133820416 21096 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 100.84
Current children cumulated vsize (KiB) 2620288

[startup+25.5009 s]
/proc/loadavg: 3.33 2.14 2.89 9/180 11427
/proc/meminfo: memFree=22690588/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 4 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
[pid=11413] ppid=11412 vsize=1889816 CPUtime=25.49 cores=0
/proc/11413/stat : 11413 (cryptominisat) R 11412 11412 11386 0 -1 4202496 31723 0 0 0 2544 5 0 0 25 0 1 0 526121265 1935171584 23475 33554432000 4194304 4587354 140734921408064 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=11415] ppid=11412 vsize=472460 CPUtime=4781.93 cores=2
/proc/11415/stat : 11415 (TNM) R 11412 11412 11386 0 -1 4202496 7210 0 0 0 478191 2 0 0 25 0 1 0 526121265 483799040 7107 33554432000 134512640 134534564 4287706240 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11415/statm: 118115 7107 95 6 0 116781 0
[pid=11416] ppid=11412 vsize=308640 CPUtime=4781.58 cores=3
/proc/11416/stat : 11416 (march_hi) R 11412 11412 11386 0 -1 4202496 71250 0 0 0 478122 36 0 0 25 0 1 0 526121265 316047360 53396 33554432000 4194304 4331988 140733887521152 18446744073709551615 4226263 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11416/statm: 77160 53396 176 34 0 75602 0
[pid=11417] ppid=11412 vsize=420896 CPUtime=19102.8 cores=4-7
/proc/11417/stat : 11417 (plingeling) S 11412 11412 11386 0 -1 4202496 2754460 1251 0 0 1909889 391 0 0 20 0 5 0 526121265 430997504 93681 33554432000 4194304 4356188 140735639286288 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11417/statm: 105224 93681 163 40 0 103768 0
[pid=11417/tid=11424] ppid=11412 vsize=420896 CPUtime=4782.19 cores=4-7
/proc/11417/task/11424/stat : 11424 (plingeling) R 11412 11412 11386 0 -1 4202560 747815 1251 0 0 478116 103 0 0 25 0 5 0 526121285 430997504 93681 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331028 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11417/tid=11425] ppid=11412 vsize=420896 CPUtime=4782.07 cores=4-7
/proc/11417/task/11425/stat : 11425 (plingeling) R 11412 11412 11386 0 -1 4202560 603973 1251 0 0 478125 82 0 0 25 0 5 0 526121285 430997504 93681 33554432000 4194304 4356188 140735639286288 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11417/tid=11426] ppid=11412 vsize=420896 CPUtime=4756.12 cores=4-7
/proc/11417/task/11426/stat : 11426 (plingeling) R 11412 11412 11386 0 -1 4202560 799572 1251 0 0 475494 118 0 0 25 0 5 0 526121285 430997504 93681 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331345 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11417/tid=11427] ppid=11412 vsize=420896 CPUtime=4782.19 cores=4-7
/proc/11417/task/11427/stat : 11427 (plingeling) R 11412 11412 11386 0 -1 4202560 594316 1251 0 0 478134 85 0 0 25 0 5 0 526121285 430997504 93681 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38231.1
Current children cumulated vsize (KiB) 3710272

[startup+4842.3 s]
/proc/loadavg: 8.06 8.03 7.97 9/180 11568
/proc/meminfo: memFree=21644252/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 4 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
[pid=11413] ppid=11412 vsize=2082444 CPUtime=4842.4 cores=0
/proc/11413/stat : 11413 (cryptominisat) R 11412 11412 11386 0 -1 4202496 1604128 0 0 0 483525 715 0 0 25 0 1 0 526121265 2132422656 87711 33554432000 4194304 4587354 140734921408064 18446744073709551615 4238616 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11413/statm: 520611 87711 337 96 0 517173 0
[pid=11414] ppid=11412 vsize=388000 CPUtime=4842.38 cores=1
/proc/11414/stat : 11414 (clasp) R 11412 11412 11386 0 -1 4202496 91800 0 0 0 484200 38 0 0 25 0 1 0 526121265 397312000 89920 33554432000 4194304 4785740 140736627916064 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11414/statm: 97000 89920 341 145 0 94045 0
[pid=11415] ppid=11412 vsize=472460 CPUtime=4841.93 cores=2
/proc/11415/stat : 11415 (TNM) R 11412 11412 11386 0 -1 4202496 7210 0 0 0 484191 2 0 0 25 0 1 0 526121265 483799040 7107 33554432000 134512640 134534564 4287706240 18446744073709551615 134530736 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11415/statm: 118115 7107 95 6 0 116781 0
[pid=11416] ppid=11412 vsize=308640 CPUtime=4841.59 cores=3
/proc/11416/stat : 11416 (march_hi) R 11412 11412 11386 0 -1 4202496 71895 0 0 0 484122 37 0 0 25 0 1 0 526121265 316047360 54041 33554432000 4194304 4331988 140733887521152 18446744073709551615 4237136 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11416/statm: 77160 54041 176 34 0 75602 0
[pid=11417] ppid=11412 vsize=453664 CPUtime=19342.5 cores=4-7
/proc/11417/stat : 11417 (plingeling) S 11412 11412 11386 0 -1 4202496 2792348 1251 0 0 1933852 396 0 0 20 0 5 0 526121265 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11417/statm: 113416 101873 163 40 0 111960 0
[pid=11417/tid=11424] ppid=11412 vsize=453664 CPUtime=4842.2 cores=4-7
/proc/11417/task/11424/stat : 11424 (plingeling) R 11412 11412 11386 0 -1 4202560 759079 1251 0 0 484115 105 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4279578 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11417/tid=11425] ppid=11412 vsize=453664 CPUtime=4842.07 cores=4-7
/proc/11417/task/11425/stat : 11425 (plingeling) R 11412 11412 11386 0 -1 4202560 610117 1251 0 0 484124 83 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4330843 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11417/tid=11426] ppid=11412 vsize=453664 CPUtime=4815.79 cores=4-7
/proc/11417/task/11426/stat : 11426 (plingeling) R 11412 11412 11386 0 -1 4202560 808788 1251 0 0 481460 119 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11417/tid=11427] ppid=11412 vsize=453664 CPUtime=4842.2 cores=4-7
/proc/11417/task/11427/stat : 11427 (plingeling) R 11412 11412 11386 0 -1 4202560 605580 1251 0 0 484133 87 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38710.8
Current children cumulated vsize (KiB) 3723532

[startup+4902.3 s]
/proc/loadavg: 8.07 8.04 7.97 9/180 11569
/proc/meminfo: memFree=21686616/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 4 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
[pid=11413] ppid=11412 vsize=2101716 CPUtime=4902.4 cores=0
/proc/11413/stat : 11413 (cryptominisat) R 11412 11412 11386 0 -1 4202496 1619066 0 0 0 489516 724 0 0 25 0 1 0 526121265 2152157184 76832 33554432000 4194304 4587354 140734921408064 18446744073709551615 4238491 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11413/statm: 525429 76832 337 96 0 521991 0
[pid=11414] ppid=11412 vsize=388000 CPUtime=4902.38 cores=1
/proc/11414/stat : 11414 (clasp) R 11412 11412 11386 0 -1 4202496 91812 0 0 0 490200 38 0 0 25 0 1 0 526121265 397312000 89932 33554432000 4194304 4785740 140736627916064 18446744073709551615 4287244 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11414/statm: 97000 89932 341 145 0 94045 0
[pid=11415] ppid=11412 vsize=472460 CPUtime=4901.92 cores=2
/proc/11415/stat : 11415 (TNM) R 11412 11412 11386 0 -1 4202496 7210 0 0 0 490190 2 0 0 25 0 1 0 526121265 483799040 7107 33554432000 134512640 134534564 4287706240 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11415/statm: 118115 7107 95 6 0 116781 0
[pid=11416] ppid=11412 vsize=308640 CPUtime=4901.59 cores=3
/proc/11416/stat : 11416 (march_hi) R 11412 11412 11386 0 -1 4202496 72512 0 0 0 490122 37 0 0 25 0 1 0 526121265 316047360 54658 33554432000 4194304 4331988 140733887521152 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11416/statm: 77160 54658 176 34 0 75602 0
[pid=11417] ppid=11412 vsize=453664 CPUtime=19582.2 cores=4-7
/proc/11417/stat : 11417 (plingeling) S 11412 11412 11386 0 -1 4202496 2827164 1251 0 0 1957815 401 0 0 20 0 5 0 526121265 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11417/statm: 113416 101873 163 40 0 111960 0
[pid=11417/tid=11424] ppid=11412 vsize=453664 CPUtime=4902.2 cores=4-7
/proc/11417/task/11424/stat : 11424 (plingeling) R 11412 11412 11386 0 -1 4202560 774439 1251 0 0 490113 107 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11417/tid=11425] ppid=11412 vsize=453664 CPUtime=4902.06 cores=4-7
/proc/11417/task/11425/stat : 11425 (plingeling) R 11412 11412 11386 0 -1 4202560 616261 1251 0 0 490122 84 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11417/tid=11426] ppid=11412 vsize=453664 CPUtime=4875.46 cores=4-7
/proc/11417/task/11426/stat : 11426 (plingeling) R 11412 11412 11386 0 -1 4202560 818004 1251 0 0 487426 120 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331268 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11417/tid=11427] ppid=11412 vsize=453664 CPUtime=4902.19 cores=4-7
/proc/11417/task/11427/stat : 11427 (plingeling) R 11412 11412 11386 0 -1 4202560 609676 1251 0 0 490132 87 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4332023 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39190.4
Current children cumulated vsize (KiB) 3742804

[startup+4962.3 s]
/proc/loadavg: 8.07 8.04 7.98 9/180 11571
/proc/meminfo: memFree=21661768/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 4 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
[pid=11413] ppid=11412 vsize=2086644 CPUtime=4962.4 cores=0
/proc/11413/stat : 11413 (cryptominisat) R 11412 11412 11386 0 -1 4202496 1637333 0 0 0 495505 735 0 0 25 0 1 0 526121265 2136723456 81829 33554432000 4194304 4587354 140734921408064 18446744073709551615 4241103 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11413/statm: 521661 81829 337 96 0 518223 0
[pid=11414] ppid=11412 vsize=389348 CPUtime=4962.38 cores=1
/proc/11414/stat : 11414 (clasp) R 11412 11412 11386 0 -1 4202496 92114 0 0 0 496200 38 0 0 25 0 1 0 526121265 398692352 90234 33554432000 4194304 4785740 140736627916064 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11414/statm: 97337 90234 341 145 0 94382 0
[pid=11415] ppid=11412 vsize=472460 CPUtime=4961.92 cores=2
/proc/11415/stat : 11415 (TNM) R 11412 11412 11386 0 -1 4202496 7210 0 0 0 496190 2 0 0 25 0 1 0 526121265 483799040 7107 33554432000 134512640 134534564 4287706240 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11415/statm: 118115 7107 95 6 0 116781 0
[pid=11416] ppid=11412 vsize=308640 CPUtime=4961.59 cores=3
/proc/11416/stat : 11416 (march_hi) R 11412 11412 11386 0 -1 4202496 73127 0 0 0 496121 38 0 0 25 0 1 0 526121265 316047360 55273 33554432000 4194304 4331988 140733887521152 18446744073709551615 4228226 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11416/statm: 77160 55273 176 34 0 75602 0
[pid=11417] ppid=11412 vsize=453664 CPUtime=19821.8 cores=4-7
/proc/11417/stat : 11417 (plingeling) S 11412 11412 11386 0 -1 4202496 2869148 1251 0 0 1981777 407 0 0 20 0 5 0 526121265 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11417/statm: 113416 101873 163 40 0 111960 0
[pid=11417/tid=11424] ppid=11412 vsize=453664 CPUtime=4962.2 cores=4-7
/proc/11417/task/11424/stat : 11424 (plingeling) R 11412 11412 11386 0 -1 4202560 786727 1251 0 0 496112 108 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11417/tid=11425] ppid=11412 vsize=453664 CPUtime=4962.06 cores=4-7
/proc/11417/task/11425/stat : 11425 (plingeling) R 11412 11412 11386 0 -1 4202560 623429 1251 0 0 496121 85 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4234297 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11417/tid=11426] ppid=11412 vsize=453664 CPUtime=4935.14 cores=4-7
/proc/11417/task/11426/stat : 11426 (plingeling) R 11412 11412 11386 0 -1 4202560 829268 1251 0 0 493392 122 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4210751 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11417/tid=11427] ppid=11412 vsize=453664 CPUtime=4962.2 cores=4-7
/proc/11417/task/11427/stat : 11427 (plingeling) R 11412 11412 11386 0 -1 4202560 620940 1251 0 0 496130 90 0 0 25 0 5 0 526121285 464551936 101873 33554432000 4194304 4356188 140735639286288 18446744073709551615 4330982 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39670.1
Current children cumulated vsize (KiB) 3729080



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.6 s]
/proc/loadavg: 8.10 8.05 7.98 9/180 11572
/proc/meminfo: memFree=21645236/32950928 swapFree=67111524/67111528
[pid=11412] ppid=11410 vsize=18324 CPUtime=0 cores=0-7
/proc/11412/stat : 11412 (ppfolio) S 11410 11412 11386 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 526121265 18763776 390 33554432000 4194304 4287625 140734370523280 18446744073709551615 274473987599 0 0 4096 65536 4456056886319382527 0 0 17 4 0 0 0
/proc/11412/statm: 4581 390 341 23 0 91 0
[pid=11413] ppid=11412 vsize=2084244 CPUtime=5003.71 cores=0
/proc/11413/stat : 11413 (cryptominisat) R 11412 11412 11386 0 -1 4202496 1648694 0 0 0 499630 741 0 0 25 0 1 0 526121265 2134265856 79785 33554432000 4194304 4587354 140734921408064 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/11413/statm: 521061 79785 337 96 0 517623 0
[pid=11414] ppid=11412 vsize=391588 CPUtime=5003.69 cores=1
/proc/11414/stat : 11414 (clasp) R 11412 11412 11386 0 -1 4202496 92742 0 0 0 500331 38 0 0 25 0 1 0 526121265 400986112 90862 33554432000 4194304 4785740 140736627916064 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/11414/statm: 97897 90862 341 145 0 94942 0
[pid=11415] ppid=11412 vsize=472460 CPUtime=5003.22 cores=2
/proc/11415/stat : 11415 (TNM) R 11412 11412 11386 0 -1 4202496 7210 0 0 0 500320 2 0 0 25 0 1 0 526121265 483799040 7107 33554432000 134512640 134534564 4287706240 18446744073709551615 134528295 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/11415/statm: 118115 7107 95 6 0 116781 0
[pid=11416] ppid=11412 vsize=308640 CPUtime=5002.9 cores=3
/proc/11416/stat : 11416 (march_hi) R 11412 11412 11386 0 -1 4202496 73592 0 0 0 500252 38 0 0 25 0 1 0 526121265 316047360 55738 33554432000 4194304 4331988 140733887521152 18446744073709551615 4237398 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/11416/statm: 77160 55738 176 34 0 75602 0
[pid=11417] ppid=11412 vsize=474144 CPUtime=19986.8 cores=4-7
/proc/11417/stat : 11417 (plingeling) S 11412 11412 11386 0 -1 4202496 2901916 1251 0 0 1998273 411 0 0 20 0 5 0 526121265 485523456 106993 33554432000 4194304 4356188 140735639286288 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/11417/statm: 118536 106993 163 40 0 117080 0
[pid=11417/tid=11424] ppid=11412 vsize=474144 CPUtime=5003.51 cores=4-7
/proc/11417/task/11424/stat : 11424 (plingeling) R 11412 11412 11386 0 -1 4202560 792871 1251 0 0 500242 109 0 0 25 0 5 0 526121285 485523456 106993 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11417/tid=11425] ppid=11412 vsize=474144 CPUtime=5003.37 cores=4-7
/proc/11417/task/11425/stat : 11425 (plingeling) R 11412 11412 11386 0 -1 4202560 633669 1251 0 0 500250 87 0 0 25 0 5 0 526121285 485523456 106993 33554432000 4194304 4356188 140735639286288 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11417/tid=11426] ppid=11412 vsize=474144 CPUtime=4976.22 cores=4-7
/proc/11417/task/11426/stat : 11426 (plingeling) R 11412 11412 11386 0 -1 4202560 840532 1251 0 0 497499 123 0 0 25 0 5 0 526121285 485523456 106993 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11417/tid=11427] ppid=11412 vsize=474144 CPUtime=5003.51 cores=4-7
/proc/11417/task/11427/stat : 11427 (plingeling) R 11412 11412 11386 0 -1 4202560 626060 1251 0 0 500261 90 0 0 25 0 5 0 526121285 485523456 106993 33554432000 4194304 4356188 140735639286288 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 3749400

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 11412 and gives
#  childrusage.ru_utime.tv_sec=5003
#  childrusage.ru_utime.tv_usec=209396
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=34994
# CPU time returned by wait4() is 5003.24
# while last known CPU time is 40000.4
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5003.62
CPU time (s): 40000.4
CPU user time (s): 39988.1
CPU system time (s): 12.3
CPU usage (%): 799.429
Max. virtual memory (cumulated for all children) (KiB): 5574140

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5003.21
system time used= 0.034994
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 7735
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 520
involuntary context switches= 14042

runsolver used 8.36773 second user time and 18.9281 second system time

The end

Launcher Data 

Begin job on node125 at 2011-04-30 08:33:36
IDJOB=3275081
IDBENCH=71408
IDSOLVER=1638
FILE ID=node125/3275081-1304145216
RUNJOBID= node125-1304145216-11400
PBS_JOBID= 13170872
Free space on /tmp= 73016 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2640c305320gyes5.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3275081-1304145216/watcher-3275081-1304145216 -o /tmp/evaluation-result-3275081-1304145216/solver-3275081-1304145216 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3275081-1304145216.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f892a2bda6fe6aec6f2fcb4e1c541df8
RANDOM SEED=699240693

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      23005128 kB
Buffers:       2132108 kB
Cached:        7224588 kB
SwapCached:          4 kB
Active:        2557548 kB
Inactive:      6862416 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      23005128 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           14048 kB
Writeback:           0 kB
AnonPages:       63504 kB
Mapped:          14368 kB
Slab:           461356 kB
PageTables:       4272 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   183672 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73008 MiB
End job on node125 at 2011-04-30 09:57:02