Trace number 3219177

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39972.1 5000.02

General information on the benchmark

Namerandom/large/
unif-k7-r85-v300-c25500-S1286362348-008-UNKNOWN.cnf
MD5SUM63b66954174098a654eacd27832fcf38
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables300
Number of clauses25500
Sum of the clauses size178500
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 525500

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3219177-1304148883/watcher-3219177-1304148883 -o /tmp/evaluation-result-3219177-1304148883/solver-3219177-1304148883 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3219177-1304148883.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.10 8.07 7.88 2/170 5417
/proc/meminfo: memFree=25132572/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 4 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0

[startup+0.0324061 s]
/proc/loadavg: 8.10 8.07 7.88 2/170 5417
/proc/meminfo: memFree=25132572/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100362 s]
/proc/loadavg: 8.10 8.07 7.88 2/170 5417
/proc/meminfo: memFree=25132572/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300322 s]
/proc/loadavg: 8.10 8.07 7.88 2/170 5417
/proc/meminfo: memFree=25132572/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700247 s]
/proc/loadavg: 8.10 8.07 7.88 2/170 5417
/proc/meminfo: memFree=25132572/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 19 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50111 s]
/proc/loadavg: 8.10 8.07 7.88 10/180 5432
/proc/meminfo: memFree=25091696/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
[pid=5418] ppid=5417 vsize=281900 CPUtime=1.49 cores=0
/proc/5418/stat : 5418 (cryptominisat) R 5417 5417 5125 0 -1 4202496 5420 0 0 0 148 1 0 0 20 0 1 0 507304137 288665600 1859 33554432000 4194304 4587354 140733250940224 18446744073709551615 4238477 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/5418/statm: 70475 1859 327 96 0 67037 0
[pid=5419] ppid=5417 vsize=20948 CPUtime=1.49 cores=1
/proc/5419/stat : 5419 (clasp) R 5417 5417 5125 0 -1 4202496 2786 0 0 0 149 0 0 0 20 0 1 0 507304137 21450752 2591 33554432000 4194304 4785740 140733872072464 18446744073709551615 4553411 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/5419/statm: 5237 2591 340 145 0 2282 0
[pid=5420] ppid=5417 vsize=455032 CPUtime=1.49 cores=2
/proc/5420/stat : 5420 (TNM) R 5417 5417 5125 0 -1 4202496 1150 0 0 0 149 0 0 0 20 0 1 0 507304137 465952768 1047 33554432000 134512640 134534564 4289166640 18446744073709551615 134524573 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/5420/statm: 113758 1047 91 6 0 112424 0
[pid=5421] ppid=5417 vsize=10932 CPUtime=1.49 cores=3
/proc/5421/stat : 5421 (march_hi) R 5417 5417 5125 0 -1 4202496 4744 0 0 0 149 0 0 0 20 0 1 0 507304138 11194368 1064 33554432000 4194304 4331988 140737339779648 18446744073709551615 4271184 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/5421/statm: 2733 1064 150 34 0 1175 0
[pid=5422] ppid=5417 vsize=125240 CPUtime=5.78 cores=4-7
/proc/5422/stat : 5422 (plingeling) S 5417 5417 5125 0 -1 4202496 5979 1249 0 0 578 0 0 0 25 0 5 0 507304138 128245760 3350 33554432000 4194304 4356188 140736444220304 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5422/statm: 31310 3350 162 40 0 29854 0
[pid=5422/tid=5429] ppid=5417 vsize=125240 CPUtime=1.44 cores=4-7
/proc/5422/task/5429/stat : 5429 (plingeling) R 5417 5417 5125 0 -1 4202560 584 1249 0 0 144 0 0 0 25 0 5 0 507304143 128245760 3350 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331045 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5422/tid=5430] ppid=5417 vsize=125240 CPUtime=1.43 cores=4-7
/proc/5422/task/5430/stat : 5430 (plingeling) R 5417 5417 5125 0 -1 4202560 567 1249 0 0 143 0 0 0 25 0 5 0 507304143 128245760 3350 33554432000 4194304 4356188 140736444220304 18446744073709551615 4279432 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5422/tid=5431] ppid=5417 vsize=125240 CPUtime=1.42 cores=4-7
/proc/5422/task/5431/stat : 5431 (plingeling) R 5417 5417 5125 0 -1 4202560 563 1249 0 0 142 0 0 0 25 0 5 0 507304143 128245760 3350 33554432000 4194304 4356188 140736444220304 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5422/tid=5432] ppid=5417 vsize=125240 CPUtime=1.43 cores=4-7
/proc/5422/task/5432/stat : 5432 (plingeling) R 5417 5417 5125 0 -1 4202560 666 1249 0 0 143 0 0 0 25 0 5 0 507304143 128245760 3350 33554432000 4194304 4356188 140736444220304 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.74
Current children cumulated vsize (KiB) 912376

[startup+3.10082 s]
/proc/loadavg: 8.10 8.07 7.88 9/180 5432
/proc/meminfo: memFree=25091448/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
[pid=5418] ppid=5417 vsize=303392 CPUtime=3.08 cores=0
/proc/5418/stat : 5418 (cryptominisat) R 5417 5417 5125 0 -1 4202496 12168 0 0 0 307 1 0 0 25 0 1 0 507304137 310673408 6504 33554432000 4194304 4587354 140733250940224 18446744073709551615 4388264 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/5418/statm: 75848 6504 337 96 0 72410 0
[pid=5419] ppid=5417 vsize=22784 CPUtime=3.08 cores=1
/proc/5419/stat : 5419 (clasp) R 5417 5417 5125 0 -1 4202496 3226 0 0 0 308 0 0 0 25 0 1 0 507304137 23330816 3031 33554432000 4194304 4785740 140733872072464 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/5419/statm: 5696 3031 340 145 0 2741 0
[pid=5420] ppid=5417 vsize=455032 CPUtime=3.09 cores=2
/proc/5420/stat : 5420 (TNM) R 5417 5417 5125 0 -1 4202496 1150 0 0 0 309 0 0 0 25 0 1 0 507304137 465952768 1047 33554432000 134512640 134534564 4289166640 18446744073709551615 134524554 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/5420/statm: 113758 1047 91 6 0 112424 0
[pid=5421] ppid=5417 vsize=14320 CPUtime=3.09 cores=3
/proc/5421/stat : 5421 (march_hi) R 5417 5417 5125 0 -1 4202496 5787 0 0 0 309 0 0 0 25 0 1 0 507304138 14663680 1732 33554432000 4194304 4331988 140737339779648 18446744073709551615 4292835 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/5421/statm: 3580 1732 174 34 0 2022 0
[pid=5422] ppid=5417 vsize=127048 CPUtime=12.19 cores=4-7
/proc/5422/stat : 5422 (plingeling) S 5417 5417 5125 0 -1 4202496 6463 1249 0 0 1218 1 0 0 25 0 5 0 507304138 130097152 3834 33554432000 4194304 4356188 140736444220304 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5422/statm: 31762 3834 162 40 0 30306 0
[pid=5422/tid=5429] ppid=5417 vsize=127048 CPUtime=3.04 cores=4-7
/proc/5422/task/5429/stat : 5429 (plingeling) R 5417 5417 5125 0 -1 4202560 665 1249 0 0 304 0 0 0 25 0 5 0 507304143 130097152 3834 33554432000 4194304 4356188 140736444220304 18446744073709551615 4279535 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5422/tid=5430] ppid=5417 vsize=127048 CPUtime=3.03 cores=4-7
/proc/5422/task/5430/stat : 5430 (plingeling) R 5417 5417 5125 0 -1 4202560 767 1249 0 0 303 0 0 0 25 0 5 0 507304143 130097152 3834 33554432000 4194304 4356188 140736444220304 18446744073709551615 4234332 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5422/tid=5431] ppid=5417 vsize=127048 CPUtime=3.02 cores=4-7
/proc/5422/task/5431/stat : 5431 (plingeling) R 5417 5417 5125 0 -1 4202560 636 1249 0 0 302 0 0 0 25 0 5 0 507304143 130097152 3834 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331119 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5422/tid=5432] ppid=5417 vsize=127048 CPUtime=3.03 cores=4-7
/proc/5422/task/5432/stat : 5432 (plingeling) R 5417 5417 5125 0 -1 4202560 796 1249 0 0 303 0 0 0 25 0 5 0 507304143 130097152 3834 33554432000 4194304 4356188 140736444220304 18446744073709551615 4273857 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.53
Current children cumulated vsize (KiB) 940900

[startup+6.30125 s]
/proc/loadavg: 8.09 8.06 7.88 9/180 5432
/proc/meminfo: memFree=25065780/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
[pid=5418] ppid=5417 vsize=293440 CPUtime=6.29 cores=0
/proc/5418/stat : 5418 (cryptominisat) R 5417 5417 5125 0 -1 4202496 12892 0 0 0 627 2 0 0 25 0 1 0 507304137 300482560 5996 33554432000 4194304 4587354 140733250940224 18446744073709551615 4237684 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/5418/statm: 73360 5996 337 96 0 69922 0
[pid=5419] ppid=5417 vsize=25576 CPUtime=6.29 cores=1
/proc/5419/stat : 5419 (clasp) R 5417 5417 5125 0 -1 4202496 3874 0 0 0 629 0 0 0 25 0 1 0 507304137 26189824 3679 33554432000 4194304 4785740 140733872072464 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/5419/statm: 6394 3679 340 145 0 3439 0
[pid=5420] ppid=5417 vsize=459288 CPUtime=6.29 cores=2
/proc/5420/stat : 5420 (TNM) R 5417 5417 5125 0 -1 4202496 1399 0 0 0 629 0 0 0 25 0 1 0 507304137 470310912 1296 33554432000 134512640 134534564 4289166640 18446744073709551615 134528504 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/5420/statm: 114822 1296 95 6 0 113488 0
[pid=5421] ppid=5417 vsize=14320 CPUtime=6.29 cores=3
/proc/5421/stat : 5421 (march_hi) R 5417 5417 5125 0 -1 4202496 5787 0 0 0 629 0 0 0 25 0 1 0 507304138 14663680 1732 33554432000 4194304 4331988 140737339779648 18446744073709551615 4226242 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/5421/statm: 3580 1732 174 34 0 2022 0
[pid=5422] ppid=5417 vsize=131004 CPUtime=24.97 cores=4-7
/proc/5422/stat : 5422 (plingeling) S 5417 5417 5125 0 -1 4202496 7452 1249 0 0 2496 1 0 0 25 0 5 0 507304138 134148096 4823 33554432000 4194304 4356188 140736444220304 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5422/statm: 32751 4823 162 40 0 31295 0
[pid=5422/tid=5429] ppid=5417 vsize=131004 CPUtime=6.24 cores=4-7
/proc/5422/task/5429/stat : 5429 (plingeling) R 5417 5417 5125 0 -1 4202560 987 1249 0 0 624 0 0 0 25 0 5 0 507304143 134148096 4823 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5422/tid=5430] ppid=5417 vsize=131004 CPUtime=6.23 cores=4-7
/proc/5422/task/5430/stat : 5430 (plingeling) R 5417 5417 5125 0 -1 4202560 1018 1249 0 0 623 0 0 0 25 0 5 0 507304143 134148096 4823 33554432000 4194304 4356188 140736444220304 18446744073709551615 4270728 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5422/tid=5431] ppid=5417 vsize=131004 CPUtime=6.2 cores=4-7
/proc/5422/task/5431/stat : 5431 (plingeling) R 5417 5417 5125 0 -1 4202560 867 1249 0 0 620 0 0 0 25 0 5 0 507304143 134148096 4823 33554432000 4194304 4356188 140736444220304 18446744073709551615 4237352 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5422/tid=5432] ppid=5417 vsize=131004 CPUtime=6.23 cores=4-7
/proc/5422/task/5432/stat : 5432 (plingeling) R 5417 5417 5125 0 -1 4202560 981 1249 0 0 623 0 0 0 25 0 5 0 507304143 134148096 4823 33554432000 4194304 4356188 140736444220304 18446744073709551615 4274159 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.13
Current children cumulated vsize (KiB) 941952

[startup+12.7011 s]
/proc/loadavg: 8.08 8.06 7.88 9/180 5432
/proc/meminfo: memFree=25061248/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
[pid=5418] ppid=5417 vsize=304900 CPUtime=12.69 cores=0
/proc/5418/stat : 5418 (cryptominisat) R 5417 5417 5125 0 -1 4202496 15146 0 0 0 1267 2 0 0 25 0 1 0 507304137 312217600 6844 33554432000 4194304 4587354 140733250940224 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/5418/statm: 76225 6844 337 96 0 72787 0
[pid=5419] ppid=5417 vsize=28484 CPUtime=12.68 cores=1
/proc/5419/stat : 5419 (clasp) R 5417 5417 5125 0 -1 4202496 4556 0 0 0 1268 0 0 0 25 0 1 0 507304137 29167616 4361 33554432000 4194304 4785740 140733872072464 18446744073709551615 4287140 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/5419/statm: 7121 4361 340 145 0 4166 0
[pid=5420] ppid=5417 vsize=459288 CPUtime=12.69 cores=2
/proc/5420/stat : 5420 (TNM) R 5417 5417 5125 0 -1 4202496 1399 0 0 0 1269 0 0 0 25 0 1 0 507304137 470310912 1296 33554432000 134512640 134534564 4289166640 18446744073709551615 134528708 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/5420/statm: 114822 1296 95 6 0 113488 0
[pid=5421] ppid=5417 vsize=14320 CPUtime=12.69 cores=3
/proc/5421/stat : 5421 (march_hi) R 5417 5417 5125 0 -1 4202496 5787 0 0 0 1269 0 0 0 25 0 1 0 507304138 14663680 1732 33554432000 4194304 4331988 140737339779648 18446744073709551615 4210459 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/5421/statm: 3580 1732 174 34 0 2022 0
[pid=5422] ppid=5417 vsize=130388 CPUtime=50.53 cores=4-7
/proc/5422/stat : 5422 (plingeling) S 5417 5417 5125 0 -1 4202496 9122 1249 0 0 5052 1 0 0 25 0 5 0 507304138 133517312 4669 33554432000 4194304 4356188 140736444220304 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5422/statm: 32597 4669 162 40 0 31141 0
[pid=5422/tid=5429] ppid=5417 vsize=130388 CPUtime=12.64 cores=4-7
/proc/5422/task/5429/stat : 5429 (plingeling) R 5417 5417 5125 0 -1 4202560 1571 1249 0 0 1264 0 0 0 25 0 5 0 507304143 133517312 4669 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5422/tid=5430] ppid=5417 vsize=130388 CPUtime=12.63 cores=4-7
/proc/5422/task/5430/stat : 5430 (plingeling) R 5417 5417 5125 0 -1 4202560 1147 1249 0 0 1263 0 0 0 25 0 5 0 507304143 133517312 4669 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5422/tid=5431] ppid=5417 vsize=130388 CPUtime=12.57 cores=4-7
/proc/5422/task/5431/stat : 5431 (plingeling) R 5417 5417 5125 0 -1 4202560 1030 1249 0 0 1257 0 0 0 25 0 5 0 507304143 133517312 4669 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331984 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5422/tid=5432] ppid=5417 vsize=130388 CPUtime=12.63 cores=4-7
/proc/5422/task/5432/stat : 5432 (plingeling) R 5417 5417 5125 0 -1 4202560 1775 1249 0 0 1263 0 0 0 25 0 5 0 507304143 133517312 4669 33554432000 4194304 4356188 140736444220304 18446744073709551615 4274252 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.28
Current children cumulated vsize (KiB) 955704

[startup+25.5008 s]
/proc/loadavg: 8.06 8.06 7.88 9/180 5432
/proc/meminfo: memFree=25039804/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
[pid=5418] ppid=5417 vsize=306756 CPUtime=25.49 cores=0
/proc/5418/stat : 5418 (cryptominisat) R 5417 5417 5125 0 -1 4202496 21014 0 0 0 2546 3 0 0 25 0 1 0 507304137 314118144 9338 33554432000 4194304 4587354 140733250940224 18446744073709551615 4238481 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=5420] ppid=5417 vsize=459288 CPUtime=4781.8 cores=2
/proc/5420/stat : 5420 (TNM) R 5417 5417 5125 0 -1 4202496 1399 0 0 0 478178 2 0 0 25 0 1 0 507304137 470310912 1296 33554432000 134512640 134534564 4289166640 18446744073709551615 134530693 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/5420/statm: 114822 1296 95 6 0 113488 0
[pid=5421] ppid=5417 vsize=691756 CPUtime=4782.41 cores=3
/proc/5421/stat : 5421 (march_hi) R 5417 5417 5125 0 -1 4202496 135730 0 0 0 478140 101 0 0 25 0 1 0 507304138 708358144 131675 33554432000 4194304 4331988 140737339779648 18446744073709551615 4299107 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/5421/statm: 172939 131675 175 34 0 171381 0
[pid=5422] ppid=5417 vsize=219592 CPUtime=19102.8 cores=4-7
/proc/5422/stat : 5422 (plingeling) S 5417 5417 5125 0 -1 4202496 402521 1249 0 0 1910215 63 0 0 25 0 5 0 507304138 224862208 26970 33554432000 4194304 4356188 140736444220304 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5422/statm: 54898 26970 162 40 0 53442 0
[pid=5422/tid=5429] ppid=5417 vsize=219592 CPUtime=4782.36 cores=4-7
/proc/5422/task/5429/stat : 5429 (plingeling) R 5417 5417 5125 0 -1 4202560 112194 1249 0 0 478220 16 0 0 25 0 5 0 507304143 224862208 26970 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5422/tid=5430] ppid=5417 vsize=219592 CPUtime=4782.36 cores=4-7
/proc/5422/task/5430/stat : 5430 (plingeling) R 5417 5417 5125 0 -1 4202560 88544 1249 0 0 478222 14 0 0 25 0 5 0 507304143 224862208 26970 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5422/tid=5431] ppid=5417 vsize=219592 CPUtime=4756.28 cores=4-7
/proc/5422/task/5431/stat : 5431 (plingeling) R 5417 5417 5125 0 -1 4202560 89492 1249 0 0 475613 15 0 0 25 0 5 0 507304143 224862208 26970 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5422/tid=5432] ppid=5417 vsize=219592 CPUtime=4781.7 cores=4-7
/proc/5422/task/5432/stat : 5432 (plingeling) R 5417 5417 5125 0 -1 4202560 108692 1249 0 0 478154 16 0 0 25 0 5 0 507304143 224862208 26970 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38231.7
Current children cumulated vsize (KiB) 2119656

[startup+4842.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/180 5573
/proc/meminfo: memFree=24031792/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
[pid=5418] ppid=5417 vsize=578312 CPUtime=4842.39 cores=0
/proc/5418/stat : 5418 (cryptominisat) R 5417 5417 5125 0 -1 4202496 1091666 0 0 0 483799 440 0 0 25 0 1 0 507304137 592191488 79389 33554432000 4194304 4587354 140733250940224 18446744073709551615 4388336 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/5418/statm: 144578 79389 337 96 0 141140 0
[pid=5419] ppid=5417 vsize=172160 CPUtime=4842.27 cores=1
/proc/5419/stat : 5419 (clasp) R 5417 5417 5125 0 -1 4202496 37726 0 0 0 484220 7 0 0 25 0 1 0 507304137 176291840 36912 33554432000 4194304 4785740 140733872072464 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/5419/statm: 43040 36912 341 145 0 40085 0
[pid=5420] ppid=5417 vsize=459288 CPUtime=4841.79 cores=2
/proc/5420/stat : 5420 (TNM) R 5417 5417 5125 0 -1 4202496 1399 0 0 0 484177 2 0 0 25 0 1 0 507304137 470310912 1296 33554432000 134512640 134534564 4289166640 18446744073709551615 134530571 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/5420/statm: 114822 1296 95 6 0 113488 0
[pid=5421] ppid=5417 vsize=697900 CPUtime=4842.41 cores=3
/proc/5421/stat : 5421 (march_hi) R 5417 5417 5125 0 -1 4202496 137429 0 0 0 484139 102 0 0 25 0 1 0 507304138 714649600 133374 33554432000 4194304 4331988 140737339779648 18446744073709551615 4239602 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/5421/statm: 174475 133374 175 34 0 172917 0
[pid=5422] ppid=5417 vsize=211528 CPUtime=19342.4 cores=4-7
/proc/5422/stat : 5422 (plingeling) S 5417 5417 5125 0 -1 4202496 406041 1249 0 0 1934181 64 0 0 25 0 5 0 507304138 216604672 24954 33554432000 4194304 4356188 140736444220304 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5422/statm: 52882 24954 162 40 0 51426 0
[pid=5422/tid=5429] ppid=5417 vsize=211528 CPUtime=4842.36 cores=4-7
/proc/5422/task/5429/stat : 5429 (plingeling) R 5417 5417 5125 0 -1 4202560 112194 1249 0 0 484220 16 0 0 25 0 5 0 507304143 216604672 24954 33554432000 4194304 4356188 140736444220304 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5422/tid=5430] ppid=5417 vsize=211528 CPUtime=4842.36 cores=4-7
/proc/5422/task/5430/stat : 5430 (plingeling) R 5417 5417 5125 0 -1 4202560 89280 1249 0 0 484222 14 0 0 25 0 5 0 507304143 216604672 24954 33554432000 4194304 4356188 140736444220304 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5422/tid=5431] ppid=5417 vsize=211528 CPUtime=4815.95 cores=4-7
/proc/5422/task/5431/stat : 5431 (plingeling) R 5417 5417 5125 0 -1 4202560 92276 1249 0 0 481580 15 0 0 25 0 5 0 507304143 216604672 24954 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331151 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5422/tid=5432] ppid=5417 vsize=211528 CPUtime=4841.7 cores=4-7
/proc/5422/task/5432/stat : 5432 (plingeling) R 5417 5417 5125 0 -1 4202560 108692 1249 0 0 484154 16 0 0 25 0 5 0 507304143 216604672 24954 33554432000 4194304 4356188 140736444220304 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38711.3
Current children cumulated vsize (KiB) 2137512

[startup+4902.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/180 5574
/proc/meminfo: memFree=24037432/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
[pid=5418] ppid=5417 vsize=578312 CPUtime=4902.39 cores=0
/proc/5418/stat : 5418 (cryptominisat) R 5417 5417 5125 0 -1 4202496 1091666 0 0 0 489799 440 0 0 25 0 1 0 507304137 592191488 79389 33554432000 4194304 4587354 140733250940224 18446744073709551615 4388345 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/5418/statm: 144578 79389 337 96 0 141140 0
[pid=5419] ppid=5417 vsize=172804 CPUtime=4902.27 cores=1
/proc/5419/stat : 5419 (clasp) R 5417 5417 5125 0 -1 4202496 37876 0 0 0 490220 7 0 0 25 0 1 0 507304137 176951296 37062 33554432000 4194304 4785740 140733872072464 18446744073709551615 4287011 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/5419/statm: 43201 37062 341 145 0 40246 0
[pid=5420] ppid=5417 vsize=459288 CPUtime=4901.79 cores=2
/proc/5420/stat : 5420 (TNM) R 5417 5417 5125 0 -1 4202496 1399 0 0 0 490177 2 0 0 25 0 1 0 507304137 470310912 1296 33554432000 134512640 134534564 4289166640 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/5420/statm: 114822 1296 95 6 0 113488 0
[pid=5421] ppid=5417 vsize=697900 CPUtime=4902.41 cores=3
/proc/5421/stat : 5421 (march_hi) R 5417 5417 5125 0 -1 4202496 139170 0 0 0 490137 104 0 0 25 0 1 0 507304138 714649600 135115 33554432000 4194304 4331988 140737339779648 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/5421/statm: 174475 135115 175 34 0 172917 0
[pid=5422] ppid=5417 vsize=197960 CPUtime=19582.1 cores=4-7
/proc/5422/stat : 5422 (plingeling) S 5417 5417 5125 0 -1 4202496 407577 1249 0 0 1958149 64 0 0 25 0 5 0 507304138 202711040 21562 33554432000 4194304 4356188 140736444220304 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5422/statm: 49490 21562 162 40 0 48034 0
[pid=5422/tid=5429] ppid=5417 vsize=197960 CPUtime=4902.37 cores=4-7
/proc/5422/task/5429/stat : 5429 (plingeling) R 5417 5417 5125 0 -1 4202560 113602 1249 0 0 490220 17 0 0 25 0 5 0 507304143 202711040 21562 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5422/tid=5430] ppid=5417 vsize=197960 CPUtime=4902.36 cores=4-7
/proc/5422/task/5430/stat : 5430 (plingeling) R 5417 5417 5125 0 -1 4202560 89344 1249 0 0 490222 14 0 0 25 0 5 0 507304143 202711040 21562 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331019 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5422/tid=5431] ppid=5417 vsize=197960 CPUtime=4875.62 cores=4-7
/proc/5422/task/5431/stat : 5431 (plingeling) R 5417 5417 5125 0 -1 4202560 92340 1249 0 0 487547 15 0 0 25 0 5 0 507304143 202711040 21562 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5422/tid=5432] ppid=5417 vsize=197960 CPUtime=4901.7 cores=4-7
/proc/5422/task/5432/stat : 5432 (plingeling) R 5417 5417 5125 0 -1 4202560 108692 1249 0 0 490154 16 0 0 25 0 5 0 507304143 202711040 21562 33554432000 4194304 4356188 140736444220304 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39191
Current children cumulated vsize (KiB) 2124588

[startup+4962.3 s]
/proc/loadavg: 8.07 8.06 8.00 9/180 5576
/proc/meminfo: memFree=24016940/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
[pid=5418] ppid=5417 vsize=578312 CPUtime=4962.39 cores=0
/proc/5418/stat : 5418 (cryptominisat) R 5417 5417 5125 0 -1 4202496 1091666 0 0 0 495799 440 0 0 25 0 1 0 507304137 592191488 79389 33554432000 4194304 4587354 140733250940224 18446744073709551615 4388336 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/5418/statm: 144578 79389 337 96 0 141140 0
[pid=5419] ppid=5417 vsize=172732 CPUtime=4962.27 cores=1
/proc/5419/stat : 5419 (clasp) R 5417 5417 5125 0 -1 4202496 38053 0 0 0 496220 7 0 0 25 0 1 0 507304137 176877568 37060 33554432000 4194304 4785740 140733872072464 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/5419/statm: 43183 37060 341 145 0 40228 0
[pid=5420] ppid=5417 vsize=459288 CPUtime=4961.79 cores=2
/proc/5420/stat : 5420 (TNM) R 5417 5417 5125 0 -1 4202496 1399 0 0 0 496177 2 0 0 25 0 1 0 507304137 470310912 1296 33554432000 134512640 134534564 4289166640 18446744073709551615 134528295 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/5420/statm: 114822 1296 95 6 0 113488 0
[pid=5421] ppid=5417 vsize=697900 CPUtime=4962.42 cores=3
/proc/5421/stat : 5421 (march_hi) R 5417 5417 5125 0 -1 4202496 140897 0 0 0 496136 106 0 0 25 0 1 0 507304138 714649600 136842 33554432000 4194304 4331988 140737339779648 18446744073709551615 4239578 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/5421/statm: 174475 136842 175 34 0 172917 0
[pid=5422] ppid=5417 vsize=212168 CPUtime=19821.8 cores=4-7
/proc/5422/stat : 5422 (plingeling) S 5417 5417 5125 0 -1 4202496 411129 1249 0 0 1982116 65 0 0 25 0 5 0 507304138 217260032 25114 33554432000 4194304 4356188 140736444220304 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5422/statm: 53042 25114 162 40 0 51586 0
[pid=5422/tid=5429] ppid=5417 vsize=212168 CPUtime=4962.36 cores=4-7
/proc/5422/task/5429/stat : 5429 (plingeling) R 5417 5417 5125 0 -1 4202560 115682 1249 0 0 496219 17 0 0 25 0 5 0 507304143 217260032 25114 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331971 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5422/tid=5430] ppid=5417 vsize=212168 CPUtime=4962.36 cores=4-7
/proc/5422/task/5430/stat : 5430 (plingeling) R 5417 5417 5125 0 -1 4202560 89408 1249 0 0 496222 14 0 0 25 0 5 0 507304143 217260032 25114 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5422/tid=5431] ppid=5417 vsize=212168 CPUtime=4935.29 cores=4-7
/proc/5422/task/5431/stat : 5431 (plingeling) R 5417 5417 5125 0 -1 4202560 92532 1249 0 0 493514 15 0 0 25 0 5 0 507304143 217260032 25114 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5422/tid=5432] ppid=5417 vsize=212168 CPUtime=4961.7 cores=4-7
/proc/5422/task/5432/stat : 5432 (plingeling) R 5417 5417 5125 0 -1 4202560 109908 1249 0 0 496154 16 0 0 25 0 5 0 507304143 217260032 25114 33554432000 4194304 4356188 140736444220304 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39670.7
Current children cumulated vsize (KiB) 2138724



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.09 8.07 8.00 9/180 5577
/proc/meminfo: memFree=24015416/32951124 swapFree=67111524/67111528
[pid=5417] ppid=5415 vsize=18324 CPUtime=0 cores=0-7
/proc/5417/stat : 5417 (ppfolio) S 5415 5417 5125 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 507304137 18763776 390 33554432000 4194304 4287625 140737288173696 18446744073709551615 233510317583 0 0 4096 65536 4604913580851068927 0 0 17 6 0 0 0
/proc/5417/statm: 4581 390 341 23 0 91 0
[pid=5418] ppid=5417 vsize=578312 CPUtime=5000.1 cores=0
/proc/5418/stat : 5418 (cryptominisat) R 5417 5417 5125 0 -1 4202496 1091666 0 0 0 499570 440 0 0 25 0 1 0 507304137 592191488 79389 33554432000 4194304 4587354 140733250940224 18446744073709551615 4388277 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/5418/statm: 144578 79389 337 96 0 141140 0
[pid=5419] ppid=5417 vsize=173016 CPUtime=4999.98 cores=1
/proc/5419/stat : 5419 (clasp) R 5417 5417 5125 0 -1 4202496 38122 0 0 0 499991 7 0 0 25 0 1 0 507304137 177168384 37129 33554432000 4194304 4785740 140733872072464 18446744073709551615 4287055 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/5419/statm: 43254 37129 341 145 0 40299 0
[pid=5420] ppid=5417 vsize=459288 CPUtime=4999.49 cores=2
/proc/5420/stat : 5420 (TNM) R 5417 5417 5125 0 -1 4202496 1399 0 0 0 499947 2 0 0 25 0 1 0 507304137 470310912 1296 33554432000 134512640 134534564 4289166640 18446744073709551615 134528680 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/5420/statm: 114822 1296 95 6 0 113488 0
[pid=5421] ppid=5417 vsize=697900 CPUtime=5000.12 cores=3
/proc/5421/stat : 5421 (march_hi) R 5417 5417 5125 0 -1 4202496 141967 0 0 0 499905 107 0 0 25 0 1 0 507304138 714649600 137912 33554432000 4194304 4331988 140737339779648 18446744073709551615 4239602 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/5421/statm: 174475 137912 175 34 0 172917 0
[pid=5422] ppid=5417 vsize=208840 CPUtime=19972.4 cores=4-7
/proc/5422/stat : 5422 (plingeling) S 5417 5417 5125 0 -1 4202496 413785 1249 0 0 1997178 65 0 0 25 0 5 0 507304138 213852160 24282 33554432000 4194304 4356188 140736444220304 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/5422/statm: 52210 24282 162 40 0 50754 0
[pid=5422/tid=5429] ppid=5417 vsize=208840 CPUtime=5000.07 cores=4-7
/proc/5422/task/5429/stat : 5429 (plingeling) R 5417 5417 5125 0 -1 4202560 115682 1249 0 0 499990 17 0 0 25 0 5 0 507304143 213852160 24282 33554432000 4194304 4356188 140736444220304 18446744073709551615 4274287 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=5422/tid=5430] ppid=5417 vsize=208840 CPUtime=5000.07 cores=4-7
/proc/5422/task/5430/stat : 5430 (plingeling) R 5417 5417 5125 0 -1 4202560 90624 1249 0 0 499993 14 0 0 25 0 5 0 507304143 213852160 24282 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5422/tid=5431] ppid=5417 vsize=208840 CPUtime=4972.8 cores=4-7
/proc/5422/task/5431/stat : 5431 (plingeling) R 5417 5417 5125 0 -1 4202560 93748 1249 0 0 497264 16 0 0 25 0 5 0 507304143 213852160 24282 33554432000 4194304 4356188 140736444220304 18446744073709551615 4331086 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=5422/tid=5432] ppid=5417 vsize=208840 CPUtime=4999.41 cores=4-7
/proc/5422/task/5432/stat : 5432 (plingeling) R 5417 5417 5125 0 -1 4202560 110132 1249 0 0 499925 16 0 0 25 0 5 0 507304143 213852160 24282 33554432000 4194304 4356188 140736444220304 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39972.1
Current children cumulated vsize (KiB) 2135680

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 5417 and gives
#  childrusage.ru_utime.tv_sec=9999
#  childrusage.ru_utime.tv_usec=388861
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=115982
# CPU time returned by wait4() is 9999.5
# while last known CPU time is 39972.1
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39972.1
CPU user time (s): 39965.9
CPU system time (s): 6.21
CPU usage (%): 799.44
Max. virtual memory (cumulated for all children) (KiB): 14019744

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 9999.39
system time used= 0.115982
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 40060
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 669
involuntary context switches= 27456

runsolver used 7.84281 second user time and 19.436 second system time

The end

Launcher Data 

Begin job on node145 at 2011-04-30 09:34:43
IDJOB=3219177
IDBENCH=82546
IDSOLVER=1638
FILE ID=node145/3219177-1304148883
RUNJOBID= node145-1304145517-5139
PBS_JOBID= 13170889
Free space on /tmp= 73028 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k7-r85-v300-c25500-S1286362348-008-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3219177-1304148883/watcher-3219177-1304148883 -o /tmp/evaluation-result-3219177-1304148883/solver-3219177-1304148883 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3219177-1304148883.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 63b66954174098a654eacd27832fcf38
RANDOM SEED=1077616131

node145.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25132852 kB
Buffers:       1287020 kB
Cached:        5936032 kB
SwapCached:          4 kB
Active:         854588 kB
Inactive:      6432088 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25132852 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           10876 kB
Writeback:           0 kB
AnonPages:       63952 kB
Mapped:          14772 kB
Slab:           466128 kB
PageTables:       4168 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   180252 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73020 MiB
End job on node145 at 2011-04-30 10:58:05