Trace number 3276291

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 40000.8 5005.02

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m52-q918.75-n50-p30-s52376212.cnf
MD5SUMf34a5dd1cdce70a322dd395bcb5f1303
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2548
Number of clauses582679
Sum of the clauses size1167806
Maximum clause length52
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2582629
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 550

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3276291-1304175716/watcher-3276291-1304175716 -o /tmp/evaluation-result-3276291-1304175716/solver-3276291-1304175716 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3276291-1304175716.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.09 8.06 8.00 1/168 20262
/proc/meminfo: memFree=30910640/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0 cores=0-7
/proc/20262/stat : 20262 (ppfolio) R 20260 20262 20071 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0

[startup+0.0906971 s]
/proc/loadavg: 8.09 8.06 8.00 1/168 20262
/proc/meminfo: memFree=30910640/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 6 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100689 s]
/proc/loadavg: 8.09 8.06 8.00 1/168 20262
/proc/meminfo: memFree=30910640/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 6 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300656 s]
/proc/loadavg: 8.09 8.06 8.00 1/168 20262
/proc/meminfo: memFree=30910640/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 0 0 0 23 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 6 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700559 s]
/proc/loadavg: 8.09 8.06 8.00 1/168 20262
/proc/meminfo: memFree=30910640/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 0 0 0 22 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 6 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50142 s]
/proc/loadavg: 7.84 8.01 7.99 9/178 20277
/proc/meminfo: memFree=30654004/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 0 0 0 22 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 6 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
[pid=20263] ppid=20262 vsize=96484 CPUtime=1.49 cores=0
/proc/20263/stat : 20263 (cryptominisat) R 20262 20262 20071 0 -1 4202496 18514 0 0 0 146 3 0 0 20 0 1 0 529170285 98799616 18399 33554432000 4194304 4587354 140735367533120 18446744073709551615 4388261 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20263/statm: 24121 18399 314 96 0 20683 0
[pid=20264] ppid=20262 vsize=48460 CPUtime=1.48 cores=1
/proc/20264/stat : 20264 (clasp) R 20262 20262 20071 0 -1 4202496 11882 0 0 0 147 1 0 0 20 0 1 0 529170285 49623040 9079 33554432000 4194304 4785740 140736030475232 18446744073709551615 4495623 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20264/statm: 12115 9079 314 145 0 9160 0
[pid=20265] ppid=20262 vsize=479188 CPUtime=1.48 cores=2
/proc/20265/stat : 20265 (TNM) R 20262 20262 20071 0 -1 4202496 9365 0 0 0 147 1 0 0 20 0 1 0 529170285 490688512 9261 33554432000 134512640 134534564 4288181968 18446744073709551615 134524510 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20265/statm: 119797 9261 91 6 0 118463 0
[pid=20266] ppid=20262 vsize=80860 CPUtime=1.48 cores=3
/proc/20266/stat : 20266 (march_hi) R 20262 20262 20071 0 -1 4202496 40094 0 0 0 144 4 0 0 20 0 1 0 529170285 82800640 9163 33554432000 4194304 4331988 140735027535584 18446744073709551615 4257036 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20266/statm: 20215 9163 159 34 0 18657 0
[pid=20267] ppid=20262 vsize=136336 CPUtime=4.88 cores=4-7
/proc/20267/stat : 20267 (plingeling) S 20262 20262 20071 0 -1 4202496 30990 1257 0 0 483 5 0 0 25 0 5 0 529170285 139608064 22510 33554432000 4194304 4356188 140737415176176 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20267/statm: 34084 22510 163 40 0 32628 0
[pid=20267/tid=20274] ppid=20262 vsize=136336 CPUtime=1.12 cores=4-7
/proc/20267/task/20274/stat : 20274 (plingeling) R 20262 20262 20071 0 -1 4202560 3057 1257 0 0 112 0 0 0 25 0 5 0 529170321 139608064 22510 33554432000 4194304 4356188 140737415176176 18446744073709551615 4211876 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20267/tid=20275] ppid=20262 vsize=136336 CPUtime=1.12 cores=4-7
/proc/20267/task/20275/stat : 20275 (plingeling) R 20262 20262 20071 0 -1 4202560 3678 1257 0 0 112 0 0 0 25 0 5 0 529170321 139608064 22510 33554432000 4194304 4356188 140737415176176 18446744073709551615 4210575 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20267/tid=20276] ppid=20262 vsize=136336 CPUtime=1.12 cores=4-7
/proc/20267/task/20276/stat : 20276 (plingeling) R 20262 20262 20071 0 -1 4202560 3427 1257 0 0 112 0 0 0 25 0 5 0 529170321 139608064 22510 33554432000 4194304 4356188 140737415176176 18446744073709551615 4210587 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20267/tid=20277] ppid=20262 vsize=136336 CPUtime=1.11 cores=4-7
/proc/20267/task/20277/stat : 20277 (plingeling) R 20262 20262 20071 0 -1 4202560 3788 1257 0 0 111 0 0 0 25 0 5 0 529170321 139608064 22510 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 10.81
Current children cumulated vsize (KiB) 859652

[startup+3.10115 s]
/proc/loadavg: 7.84 8.01 7.99 9/178 20277
/proc/meminfo: memFree=30619160/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 6 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
[pid=20263] ppid=20262 vsize=110224 CPUtime=3.09 cores=0
/proc/20263/stat : 20263 (cryptominisat) R 20262 20262 20071 0 -1 4202496 33792 0 0 0 303 6 0 0 25 0 1 0 529170285 112869376 21799 33554432000 4194304 4587354 140735367533120 18446744073709551615 4237643 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20263/statm: 27556 21799 326 96 0 24118 0
[pid=20264] ppid=20262 vsize=48460 CPUtime=3.08 cores=1
/proc/20264/stat : 20264 (clasp) R 20262 20262 20071 0 -1 4202496 11901 0 0 0 307 1 0 0 25 0 1 0 529170285 49623040 9098 33554432000 4194304 4785740 140736030475232 18446744073709551615 4483461 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20264/statm: 12115 9098 317 145 0 9160 0
[pid=20265] ppid=20262 vsize=479188 CPUtime=3.08 cores=2
/proc/20265/stat : 20265 (TNM) R 20262 20262 20071 0 -1 4202496 9365 0 0 0 307 1 0 0 25 0 1 0 529170285 490688512 9261 33554432000 134512640 134534564 4288181968 18446744073709551615 134524554 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20265/statm: 119797 9261 91 6 0 118463 0
[pid=20266] ppid=20262 vsize=83376 CPUtime=3.09 cores=3
/proc/20266/stat : 20266 (march_hi) R 20262 20262 20071 0 -1 4202496 45411 0 0 0 304 5 0 0 25 0 1 0 529170285 85377024 9818 33554432000 4194304 4331988 140735027535584 18446744073709551615 4226537 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20266/statm: 20844 9818 171 34 0 19286 0
[pid=20267] ppid=20262 vsize=149648 CPUtime=11.27 cores=4-7
/proc/20267/stat : 20267 (plingeling) S 20262 20262 20071 0 -1 4202496 33295 1257 0 0 1122 5 0 0 25 0 5 0 529170285 153239552 24815 33554432000 4194304 4356188 140737415176176 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20267/statm: 37412 24815 163 40 0 35956 0
[pid=20267/tid=20274] ppid=20262 vsize=149648 CPUtime=2.72 cores=4-7
/proc/20267/task/20274/stat : 20274 (plingeling) R 20262 20262 20071 0 -1 4202560 3057 1257 0 0 272 0 0 0 25 0 5 0 529170321 153239552 24815 33554432000 4194304 4356188 140737415176176 18446744073709551615 4212207 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20267/tid=20275] ppid=20262 vsize=149648 CPUtime=2.72 cores=4-7
/proc/20267/task/20275/stat : 20275 (plingeling) R 20262 20262 20071 0 -1 4202560 3678 1257 0 0 272 0 0 0 25 0 5 0 529170321 153239552 24815 33554432000 4194304 4356188 140737415176176 18446744073709551615 4212992 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20267/tid=20276] ppid=20262 vsize=149648 CPUtime=2.72 cores=4-7
/proc/20267/task/20276/stat : 20276 (plingeling) R 20262 20262 20071 0 -1 4202560 3427 1257 0 0 272 0 0 0 25 0 5 0 529170321 153239552 24815 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20267/tid=20277] ppid=20262 vsize=149648 CPUtime=2.72 cores=4-7
/proc/20267/task/20277/stat : 20277 (plingeling) R 20262 20262 20071 0 -1 4202560 6093 1257 0 0 271 1 0 0 25 0 5 0 529170321 153239552 24815 33554432000 4194304 4356188 140737415176176 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 23.61
Current children cumulated vsize (KiB) 889220

[startup+6.30059 s]
/proc/loadavg: 8.09 8.06 8.00 10/178 20277
/proc/meminfo: memFree=30583944/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0.05 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 5 0 0 15 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 6 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
[pid=20263] ppid=20262 vsize=1147600 CPUtime=6.29 cores=0
/proc/20263/stat : 20263 (cryptominisat) R 20262 20262 20071 0 -1 4202496 36620 0 0 0 623 6 0 0 25 0 1 0 529170285 1175142400 22209 33554432000 4194304 4587354 140735367533120 18446744073709551615 4238538 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20263/statm: 286900 22209 331 96 0 283462 0
[pid=20264] ppid=20262 vsize=46420 CPUtime=6.29 cores=1
/proc/20264/stat : 20264 (clasp) R 20262 20262 20071 0 -1 4202496 12769 0 0 0 627 2 0 0 25 0 1 0 529170285 47534080 8827 33554432000 4194304 4785740 140736030475232 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20264/statm: 11605 8827 341 145 0 8650 0
[pid=20265] ppid=20262 vsize=479188 CPUtime=6.28 cores=2
/proc/20265/stat : 20265 (TNM) R 20262 20262 20071 0 -1 4202496 9365 0 0 0 627 1 0 0 25 0 1 0 529170285 490688512 9261 33554432000 134512640 134534564 4288181968 18446744073709551615 134524508 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20265/statm: 119797 9261 91 6 0 118463 0
[pid=20266] ppid=20262 vsize=83424 CPUtime=6.29 cores=3
/proc/20266/stat : 20266 (march_hi) R 20262 20262 20071 0 -1 4202496 45451 0 0 0 624 5 0 0 25 0 1 0 529170285 85426176 9858 33554432000 4194304 4331988 140735027535584 18446744073709551615 4226438 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20266/statm: 20856 9858 171 34 0 19298 0
[pid=20267] ppid=20262 vsize=161168 CPUtime=22.92 cores=4-7
/proc/20267/stat : 20267 (plingeling) S 20262 20262 20071 0 -1 4202496 41294 1257 0 0 2280 12 0 0 25 0 5 0 529170285 165036032 28750 33554432000 4194304 4356188 140737415176176 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20267/statm: 40292 28750 163 40 0 38836 0
[pid=20267/tid=20274] ppid=20262 vsize=161168 CPUtime=5.92 cores=4-7
/proc/20267/task/20274/stat : 20274 (plingeling) R 20262 20262 20071 0 -1 4202560 5234 1257 0 0 592 0 0 0 25 0 5 0 529170321 165036032 28750 33554432000 4194304 4356188 140737415176176 18446744073709551615 4279968 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20267/tid=20275] ppid=20262 vsize=161168 CPUtime=4.8 cores=4-7
/proc/20267/task/20275/stat : 20275 (plingeling) R 20262 20262 20071 0 -1 4202560 4126 1257 0 0 474 6 0 0 25 0 5 0 529170321 165036032 28750 33554432000 4194304 4356188 140737415176176 18446744073709551615 4234297 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20267/tid=20276] ppid=20262 vsize=161168 CPUtime=5.92 cores=4-7
/proc/20267/task/20276/stat : 20276 (plingeling) R 20262 20262 20071 0 -1 4202560 5474 1257 0 0 591 1 0 0 25 0 5 0 529170321 165036032 28750 33554432000 4194304 4356188 140737415176176 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20267/tid=20277] ppid=20262 vsize=161168 CPUtime=5.89 cores=4-7
/proc/20267/task/20277/stat : 20277 (plingeling) R 20262 20262 20071 0 -1 4202560 9420 1257 0 0 588 1 0 0 25 0 5 0 529170321 165036032 28750 33554432000 4194304 4356188 140737415176176 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 48.12
Current children cumulated vsize (KiB) 1936124

[startup+12.7005 s]
/proc/loadavg: 8.08 8.06 8.00 9/178 20277
/proc/meminfo: memFree=30544580/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0.05 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 5 0 0 15 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 0 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
[pid=20263] ppid=20262 vsize=1147600 CPUtime=12.69 cores=0
/proc/20263/stat : 20263 (cryptominisat) R 20262 20262 20071 0 -1 4202496 36636 0 0 0 1263 6 0 0 25 0 1 0 529170285 1175142400 22225 33554432000 4194304 4587354 140735367533120 18446744073709551615 4365352 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20263/statm: 286900 22225 337 96 0 283462 0
[pid=20264] ppid=20262 vsize=67700 CPUtime=12.68 cores=1
/proc/20264/stat : 20264 (clasp) R 20262 20262 20071 0 -1 4202496 18131 0 0 0 1265 3 0 0 25 0 1 0 529170285 69324800 14189 33554432000 4194304 4785740 140736030475232 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20264/statm: 16925 14189 341 145 0 13970 0
[pid=20265] ppid=20262 vsize=483808 CPUtime=12.68 cores=2
/proc/20265/stat : 20265 (TNM) R 20262 20262 20071 0 -1 4202496 13574 0 0 0 1267 1 0 0 25 0 1 0 529170285 495419392 13470 33554432000 134512640 134534564 4288181968 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20265/statm: 120952 13470 95 6 0 119618 0
[pid=20266] ppid=20262 vsize=84032 CPUtime=12.69 cores=3
/proc/20266/stat : 20266 (march_hi) R 20262 20262 20071 0 -1 4202496 45597 0 0 0 1264 5 0 0 25 0 1 0 529170285 86048768 10004 33554432000 4194304 4331988 140735027535584 18446744073709551615 4246719 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20266/statm: 21008 10004 174 34 0 19450 0
[pid=20267] ppid=20262 vsize=185232 CPUtime=44.6 cores=4-7
/proc/20267/stat : 20267 (plingeling) S 20262 20262 20071 0 -1 4202496 46255 1257 0 0 4442 18 0 0 25 0 5 0 529170285 189677568 33711 33554432000 4194304 4356188 140737415176176 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20267/statm: 46308 33711 163 40 0 44852 0
[pid=20267/tid=20274] ppid=20262 vsize=185232 CPUtime=12.33 cores=4-7
/proc/20267/task/20274/stat : 20274 (plingeling) R 20262 20262 20071 0 -1 4202560 6738 1257 0 0 1232 1 0 0 25 0 5 0 529170321 189677568 33711 33554432000 4194304 4356188 140737415176176 18446744073709551615 4287099 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20267/tid=20275] ppid=20262 vsize=185232 CPUtime=9.11 cores=4-7
/proc/20267/task/20275/stat : 20275 (plingeling) R 20262 20262 20071 0 -1 4202560 6047 1257 0 0 902 9 0 0 25 0 5 0 529170321 189677568 33711 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331028 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20267/tid=20276] ppid=20262 vsize=185232 CPUtime=12.31 cores=4-7
/proc/20267/task/20276/stat : 20276 (plingeling) R 20262 20262 20071 0 -1 4202560 5474 1257 0 0 1230 1 0 0 25 0 5 0 529170321 189677568 33711 33554432000 4194304 4356188 140737415176176 18446744073709551615 4330931 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20267/tid=20277] ppid=20262 vsize=185232 CPUtime=10.45 cores=4-7
/proc/20267/task/20277/stat : 20277 (plingeling) R 20262 20262 20071 0 -1 4202560 10956 1257 0 0 1042 3 0 0 25 0 5 0 529170321 189677568 33711 33554432000 4194304 4356188 140737415176176 18446744073709551615 4279448 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 95.39
Current children cumulated vsize (KiB) 1986696

[startup+25.5013 s]
/proc/loadavg: 8.07 8.06 8.00 9/178 20277
/proc/meminfo: memFree=30444144/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0.05 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 5 0 0 15 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 0 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
[pid=20263] ppid=20262 vsize=1170564 CPUtime=25.49 cores=0
/proc/20263/stat : 20263 (cryptominisat) R 20262 20262 20071 0 -1 4202496 41044 0 0 0 2543 6 0 0 25 0 1 0 529170285 1198657536 26633 33554432000 4194304 4587354 140735367533120 18446744073709551615 4365361 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=20265] ppid=20262 vsize=483808 CPUtime=4781.62 cores=2
/proc/20265/stat : 20265 (TNM) R 20262 20262 20071 0 -1 4202496 13574 0 0 0 478160 2 0 0 25 0 1 0 529170285 495419392 13470 33554432000 134512640 134534564 4288181968 18446744073709551615 134530124 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20265/statm: 120952 13470 95 6 0 119618 0
[pid=20266] ppid=20262 vsize=87116 CPUtime=4781.88 cores=3
/proc/20266/stat : 20266 (march_hi) R 20262 20262 20071 0 -1 4202496 46247 0 0 0 478183 5 0 0 25 0 1 0 529170285 89206784 10654 33554432000 4194304 4331988 140735027535584 18446744073709551615 4228235 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20266/statm: 21779 10654 174 34 0 20221 0
[pid=20267] ppid=20262 vsize=918684 CPUtime=19093.2 cores=4-7
/proc/20267/stat : 20267 (plingeling) S 20262 20262 20071 0 -1 4202496 1870321 1257 0 0 1908928 390 0 0 25 0 5 0 529170285 940732416 218097 33554432000 4194304 4356188 140737415176176 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20267/statm: 229671 218097 163 40 0 228215 0
[pid=20267/tid=20274] ppid=20262 vsize=918684 CPUtime=4782 cores=4-7
/proc/20267/task/20274/stat : 20274 (plingeling) R 20262 20262 20071 0 -1 4202560 664604 1257 0 0 478105 95 0 0 25 0 5 0 529170321 940732416 218097 33554432000 4194304 4356188 140737415176176 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20267/tid=20275] ppid=20262 vsize=918684 CPUtime=4777.98 cores=4-7
/proc/20267/task/20275/stat : 20275 (plingeling) R 20262 20262 20071 0 -1 4202560 266083 1257 0 0 477747 51 0 0 25 0 5 0 529170321 940732416 218097 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20267/tid=20276] ppid=20262 vsize=918684 CPUtime=4778.77 cores=4-7
/proc/20267/task/20276/stat : 20276 (plingeling) R 20262 20262 20071 0 -1 4202560 290832 1257 0 0 477719 158 0 0 25 0 5 0 529170321 940732416 218097 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331984 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20267/tid=20277] ppid=20262 vsize=918684 CPUtime=4754.03 cores=4-7
/proc/20267/task/20277/stat : 20277 (plingeling) R 20262 20262 20071 0 -1 4202560 631762 1257 0 0 475321 82 0 0 25 0 5 0 529170321 940732416 218097 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331268 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38220.4
Current children cumulated vsize (KiB) 4199108

[startup+4842.3 s]
/proc/loadavg: 8.05 8.04 8.00 9/178 20421
/proc/meminfo: memFree=28699164/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0.05 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 5 0 0 15 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 0 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
[pid=20263] ppid=20262 vsize=1441892 CPUtime=4842.37 cores=0
/proc/20263/stat : 20263 (cryptominisat) R 20262 20262 20071 0 -1 4202496 1004919 0 0 0 483783 454 0 0 25 0 1 0 529170285 1476497408 99015 33554432000 4194304 4587354 140735367533120 18446744073709551615 4238578 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20263/statm: 360473 99015 337 96 0 357035 0
[pid=20264] ppid=20262 vsize=904300 CPUtime=4841.35 cores=1
/proc/20264/stat : 20264 (clasp) R 20262 20262 20071 0 -1 4202496 217349 0 0 0 484047 88 0 0 25 0 1 0 529170285 926003200 213407 33554432000 4194304 4785740 140736030475232 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20264/statm: 226075 213407 341 145 0 223120 0
[pid=20265] ppid=20262 vsize=483808 CPUtime=4841.61 cores=2
/proc/20265/stat : 20265 (TNM) R 20262 20262 20071 0 -1 4202496 13574 0 0 0 484159 2 0 0 25 0 1 0 529170285 495419392 13470 33554432000 134512640 134534564 4288181968 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20265/statm: 120952 13470 95 6 0 119618 0
[pid=20266] ppid=20262 vsize=87116 CPUtime=4841.85 cores=3
/proc/20266/stat : 20266 (march_hi) R 20262 20262 20071 0 -1 4202496 46247 0 0 0 484180 5 0 0 25 0 1 0 529170285 89206784 10654 33554432000 4194304 4331988 140735027535584 18446744073709551615 4228235 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20266/statm: 21779 10654 174 34 0 20221 0
[pid=20267] ppid=20262 vsize=910492 CPUtime=19332.8 cores=4-7
/proc/20267/stat : 20267 (plingeling) S 20262 20262 20071 0 -1 4202496 1886705 1257 0 0 1932892 393 0 0 25 0 5 0 529170285 932343808 216049 33554432000 4194304 4356188 140737415176176 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20267/statm: 227623 216049 163 40 0 226167 0
[pid=20267/tid=20274] ppid=20262 vsize=910492 CPUtime=4842 cores=4-7
/proc/20267/task/20274/stat : 20274 (plingeling) R 20262 20262 20071 0 -1 4202560 667676 1257 0 0 484105 95 0 0 25 0 5 0 529170321 932343808 216049 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20267/tid=20275] ppid=20262 vsize=910492 CPUtime=4837.98 cores=4-7
/proc/20267/task/20275/stat : 20275 (plingeling) R 20262 20262 20071 0 -1 4202560 268131 1257 0 0 483747 51 0 0 25 0 5 0 529170321 932343808 216049 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20267/tid=20276] ppid=20262 vsize=910492 CPUtime=4838.77 cores=4-7
/proc/20267/task/20276/stat : 20276 (plingeling) R 20262 20262 20071 0 -1 4202560 292880 1257 0 0 483718 159 0 0 25 0 5 0 529170321 932343808 216049 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20267/tid=20277] ppid=20262 vsize=910492 CPUtime=4813.7 cores=4-7
/proc/20267/task/20277/stat : 20277 (plingeling) R 20262 20262 20071 0 -1 4202560 640978 1257 0 0 481287 83 0 0 25 0 5 0 529170321 932343808 216049 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38700.1
Current children cumulated vsize (KiB) 3845932

[startup+4902.3 s]
/proc/loadavg: 8.05 8.05 8.00 9/178 20422
/proc/meminfo: memFree=28782964/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0.05 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 5 0 0 15 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 0 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
[pid=20263] ppid=20262 vsize=1441872 CPUtime=4902.38 cores=0
/proc/20263/stat : 20263 (cryptominisat) R 20262 20262 20071 0 -1 4202496 1015232 0 0 0 489777 461 0 0 25 0 1 0 529170285 1476476928 99015 33554432000 4194304 4587354 140735367533120 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20263/statm: 360468 99015 337 96 0 357030 0
[pid=20264] ppid=20262 vsize=907180 CPUtime=4901.35 cores=1
/proc/20264/stat : 20264 (clasp) R 20262 20262 20071 0 -1 4202496 218032 0 0 0 490047 88 0 0 25 0 1 0 529170285 928952320 214090 33554432000 4194304 4785740 140736030475232 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20264/statm: 226795 214090 341 145 0 223840 0
[pid=20265] ppid=20262 vsize=483808 CPUtime=4901.6 cores=2
/proc/20265/stat : 20265 (TNM) R 20262 20262 20071 0 -1 4202496 13574 0 0 0 490158 2 0 0 25 0 1 0 529170285 495419392 13470 33554432000 134512640 134534564 4288181968 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20265/statm: 120952 13470 95 6 0 119618 0
[pid=20266] ppid=20262 vsize=87116 CPUtime=4901.82 cores=3
/proc/20266/stat : 20266 (march_hi) R 20262 20262 20071 0 -1 4202496 46247 0 0 0 490177 5 0 0 25 0 1 0 529170285 89206784 10654 33554432000 4194304 4331988 140735027535584 18446744073709551615 4226443 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20266/statm: 21779 10654 174 34 0 20221 0
[pid=20267] ppid=20262 vsize=812188 CPUtime=19572.5 cores=4-7
/proc/20267/stat : 20267 (plingeling) S 20262 20262 20071 0 -1 4202496 1911281 1257 0 0 1956853 398 0 0 25 0 5 0 529170285 831680512 191473 33554432000 4194304 4356188 140737415176176 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20267/statm: 203047 191473 163 40 0 201591 0
[pid=20267/tid=20274] ppid=20262 vsize=812188 CPUtime=4902.01 cores=4-7
/proc/20267/task/20274/stat : 20274 (plingeling) R 20262 20262 20071 0 -1 4202560 680988 1257 0 0 490104 97 0 0 25 0 5 0 529170321 831680512 191473 33554432000 4194304 4356188 140737415176176 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20267/tid=20275] ppid=20262 vsize=812188 CPUtime=4897.98 cores=4-7
/proc/20267/task/20275/stat : 20275 (plingeling) R 20262 20262 20071 0 -1 4202560 273251 1257 0 0 489746 52 0 0 25 0 5 0 529170321 831680512 191473 33554432000 4194304 4356188 140737415176176 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20267/tid=20276] ppid=20262 vsize=812188 CPUtime=4898.76 cores=4-7
/proc/20267/task/20276/stat : 20276 (plingeling) R 20262 20262 20071 0 -1 4202560 292880 1257 0 0 489715 161 0 0 25 0 5 0 529170321 831680512 191473 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331296 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20267/tid=20277] ppid=20262 vsize=812188 CPUtime=4873.38 cores=4-7
/proc/20267/task/20277/stat : 20277 (plingeling) R 20262 20262 20071 0 -1 4202560 647122 1257 0 0 487254 84 0 0 25 0 5 0 529170321 831680512 191473 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331337 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39179.7
Current children cumulated vsize (KiB) 3750488

[startup+4962.3 s]
/proc/loadavg: 8.02 8.03 8.00 9/178 20424
/proc/meminfo: memFree=28626580/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0.05 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 5 0 0 15 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 0 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
[pid=20263] ppid=20262 vsize=1441872 CPUtime=4962.38 cores=0
/proc/20263/stat : 20263 (cryptominisat) R 20262 20262 20071 0 -1 4202496 1024098 0 0 0 495772 466 0 0 25 0 1 0 529170285 1476476928 107786 33554432000 4194304 4587354 140735367533120 18446744073709551615 4236866 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20263/statm: 360468 107786 337 96 0 357030 0
[pid=20264] ppid=20262 vsize=909580 CPUtime=4961.36 cores=1
/proc/20264/stat : 20264 (clasp) R 20262 20262 20071 0 -1 4202496 218654 0 0 0 496047 89 0 0 25 0 1 0 529170285 931409920 214712 33554432000 4194304 4785740 140736030475232 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20264/statm: 227395 214712 341 145 0 224440 0
[pid=20265] ppid=20262 vsize=483808 CPUtime=4961.6 cores=2
/proc/20265/stat : 20265 (TNM) R 20262 20262 20071 0 -1 4202496 13574 0 0 0 496158 2 0 0 25 0 1 0 529170285 495419392 13470 33554432000 134512640 134534564 4288181968 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20265/statm: 120952 13470 95 6 0 119618 0
[pid=20266] ppid=20262 vsize=87116 CPUtime=4961.8 cores=3
/proc/20266/stat : 20266 (march_hi) R 20262 20262 20071 0 -1 4202496 46247 0 0 0 496175 5 0 0 25 0 1 0 529170285 89206784 10654 33554432000 4194304 4331988 140735027535584 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20266/statm: 21779 10654 174 34 0 20221 0
[pid=20267] ppid=20262 vsize=943260 CPUtime=19812.2 cores=4-7
/proc/20267/stat : 20267 (plingeling) S 20262 20262 20071 0 -1 4202496 1963505 1257 0 0 1980813 406 0 0 25 0 5 0 529170285 965898240 224241 33554432000 4194304 4356188 140737415176176 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20267/statm: 235815 224241 163 40 0 234359 0
[pid=20267/tid=20274] ppid=20262 vsize=943260 CPUtime=4962.01 cores=4-7
/proc/20267/task/20274/stat : 20274 (plingeling) R 20262 20262 20071 0 -1 4202560 695324 1257 0 0 496102 99 0 0 25 0 5 0 529170321 965898240 224241 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331971 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20267/tid=20275] ppid=20262 vsize=943260 CPUtime=4957.99 cores=4-7
/proc/20267/task/20275/stat : 20275 (plingeling) R 20262 20262 20071 0 -1 4202560 276323 1257 0 0 495746 53 0 0 25 0 5 0 529170321 965898240 224241 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20267/tid=20276] ppid=20262 vsize=943260 CPUtime=4958.76 cores=4-7
/proc/20267/task/20276/stat : 20276 (plingeling) R 20262 20262 20071 0 -1 4202560 303120 1257 0 0 495713 163 0 0 25 0 5 0 529170321 965898240 224241 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20267/tid=20277] ppid=20262 vsize=943260 CPUtime=4933.05 cores=4-7
/proc/20267/task/20277/stat : 20277 (plingeling) R 20262 20262 20071 0 -1 4202560 671698 1257 0 0 493218 87 0 0 25 0 5 0 529170321 965898240 224241 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39659.4
Current children cumulated vsize (KiB) 3883960



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5005.01 s]
/proc/loadavg: 8.05 8.04 8.00 10/178 20425
/proc/meminfo: memFree=28744492/32951132 swapFree=67111364/67111528
[pid=20262] ppid=20260 vsize=18324 CPUtime=0.05 cores=0-7
/proc/20262/stat : 20262 (ppfolio) S 20260 20262 20071 0 -1 4202496 521 0 0 0 0 5 0 0 15 0 1 0 529170284 18763776 390 33554432000 4194304 4287625 140735895967392 18446744073709551615 242970570255 0 0 4096 65536 7776679446420914175 0 0 17 0 0 0 0
/proc/20262/statm: 4581 390 341 23 0 91 0
[pid=20263] ppid=20262 vsize=1441868 CPUtime=5005.09 cores=0
/proc/20263/stat : 20263 (cryptominisat) R 20262 20262 20071 0 -1 4202496 1028229 0 0 0 500040 469 0 0 25 0 1 0 529170285 1476472832 101592 33554432000 4194304 4587354 140735367533120 18446744073709551615 4238421 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20263/statm: 360467 101592 337 96 0 357029 0
[pid=20264] ppid=20262 vsize=911020 CPUtime=5004.07 cores=1
/proc/20264/stat : 20264 (clasp) R 20262 20262 20071 0 -1 4202496 219004 0 0 0 500318 89 0 0 25 0 1 0 529170285 932884480 215062 33554432000 4194304 4785740 140736030475232 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20264/statm: 227755 215062 341 145 0 224800 0
[pid=20265] ppid=20262 vsize=483808 CPUtime=5004.31 cores=2
/proc/20265/stat : 20265 (TNM) R 20262 20262 20071 0 -1 4202496 13574 0 0 0 500429 2 0 0 25 0 1 0 529170285 495419392 13470 33554432000 134512640 134534564 4288181968 18446744073709551615 134530573 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20265/statm: 120952 13470 95 6 0 119618 0
[pid=20266] ppid=20262 vsize=87116 CPUtime=5004.49 cores=3
/proc/20266/stat : 20266 (march_hi) R 20262 20262 20071 0 -1 4202496 46247 0 0 0 500444 5 0 0 25 0 1 0 529170285 89206784 10654 33554432000 4194304 4331988 140735027535584 18446744073709551615 4228235 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20266/statm: 21779 10654 174 34 0 20221 0
[pid=20267] ppid=20262 vsize=849052 CPUtime=19982.8 cores=4-7
/proc/20267/stat : 20267 (plingeling) S 20262 20262 20071 0 -1 4202496 1976817 1257 0 0 1997870 409 0 0 25 0 5 0 529170285 869429248 200689 33554432000 4194304 4356188 140737415176176 18446744073709551615 242982352869 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20267/statm: 212263 200689 163 40 0 210807 0
[pid=20267/tid=20274] ppid=20262 vsize=849052 CPUtime=5004.72 cores=4-7
/proc/20267/task/20274/stat : 20274 (plingeling) R 20262 20262 20071 0 -1 4202560 697372 1257 0 0 500373 99 0 0 25 0 5 0 529170321 869429248 200689 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20267/tid=20275] ppid=20262 vsize=849052 CPUtime=5000.7 cores=4-7
/proc/20267/task/20275/stat : 20275 (plingeling) R 20262 20262 20071 0 -1 4202560 279395 1257 0 0 500016 54 0 0 25 0 5 0 529170321 869429248 200689 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331028 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20267/tid=20276] ppid=20262 vsize=849052 CPUtime=5001.47 cores=4-7
/proc/20267/task/20276/stat : 20276 (plingeling) R 20262 20262 20071 0 -1 4202560 311312 1257 0 0 499982 165 0 0 25 0 5 0 529170321 869429248 200689 33554432000 4194304 4356188 140737415176176 18446744073709551615 4331141 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20267/tid=20277] ppid=20262 vsize=849052 CPUtime=4975.52 cores=4-7
/proc/20267/task/20277/stat : 20277 (plingeling) R 20262 20262 20071 0 -1 4202560 671698 1257 0 0 497465 87 0 0 25 0 5 0 529170321 869429248 200689 33554432000 4194304 4356188 140737415176176 18446744073709551615 4297533 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 3791188

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 20262 and gives
#  childrusage.ru_utime.tv_sec=10008
#  childrusage.ru_utime.tv_usec=750438
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=141978
# CPU time returned by wait4() is 10008.9
# while last known CPU time is 40000.8
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5005.02
CPU time (s): 40000.8
CPU user time (s): 39991
CPU system time (s): 9.79
CPU usage (%): 799.213
Max. virtual memory (cumulated for all children) (KiB): 4884340

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 10008.8
system time used= 0.141978
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 60344
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 563
involuntary context switches= 27571

runsolver used 7.53285 second user time and 19.772 second system time

The end

Launcher Data 

Begin job on node128 at 2011-04-30 17:01:56
IDJOB=3276291
IDBENCH=88204
IDSOLVER=1638
FILE ID=node128/3276291-1304175716
RUNJOBID= node128-1304174494-20085
PBS_JOBID= 13170971
Free space on /tmp= 67524 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m52-q918.75-n50-p30-s52376212.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3276291-1304175716/watcher-3276291-1304175716 -o /tmp/evaluation-result-3276291-1304175716/solver-3276291-1304175716 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3276291-1304175716.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f34a5dd1cdce70a322dd395bcb5f1303
RANDOM SEED=1864513265

node128.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5434.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      30911044 kB
Buffers:        463104 kB
Cached:        1095964 kB
SwapCached:          0 kB
Active:         190860 kB
Inactive:      1432224 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      30911044 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:           17484 kB
Writeback:           0 kB
AnonPages:       64088 kB
Mapped:          15064 kB
Slab:           352820 kB
PageTables:       4908 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS:   189580 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 67512 MiB
End job on node128 at 2011-04-30 18:25:23