Trace number 3310331

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 40000.2 5003.52

General information on the benchmark

Nameapplication/kullmann/AES/Benchmarks/
64/aes_64_2_keyfind_1.cnf
MD5SUM1024a3346322fdba0a20b51f99eae4c0
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1000
Number of clauses5176
Sum of the clauses size20024
Maximum clause length5
Minimum clause length1
Number of clauses of size 1136
Number of clauses of size 2160
Number of clauses of size 32080
Number of clauses of size 4672
Number of clauses of size 52128
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3310331-1304012763/watcher-3310331-1304012763 -o /tmp/evaluation-result-3310331-1304012763/solver-3310331-1304012763 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3310331-1304012763.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.05 8.04 8.00 1/170 20780
/proc/meminfo: memFree=11238604/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) S 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 25 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 4334345759487950847 0 0 17 4 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0

[startup+0.148561 s]
/proc/loadavg: 8.05 8.04 8.00 1/170 20780
/proc/meminfo: memFree=11238604/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) S 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 25 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 4334345759487950847 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.200551 s]
/proc/loadavg: 8.05 8.04 8.00 1/170 20780
/proc/meminfo: memFree=11238604/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) R 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 24 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300533 s]
/proc/loadavg: 8.05 8.04 8.00 1/170 20780
/proc/meminfo: memFree=11238604/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) R 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 24 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700462 s]
/proc/loadavg: 8.05 8.04 8.00 1/170 20780
/proc/meminfo: memFree=11238604/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) R 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 21 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50132 s]
/proc/loadavg: 8.05 8.04 8.00 9/180 20795
/proc/meminfo: memFree=11218560/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) S 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 4334345759487950847 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
[pid=20781] ppid=20780 vsize=461856 CPUtime=1.48 cores=0
/proc/20781/stat : 20781 (cryptominisat) R 20780 20780 20191 0 -1 4202496 4481 0 0 0 148 0 0 0 20 0 1 0 493692169 472940544 2889 33554432000 4194304 4587354 140736753186480 18446744073709551615 4364904 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20781/statm: 115464 2889 338 96 0 112026 0
[pid=20782] ppid=20780 vsize=14768 CPUtime=1.49 cores=1
/proc/20782/stat : 20782 (clasp) R 20780 20780 20191 0 -1 4202496 1145 0 0 0 149 0 0 0 20 0 1 0 493692169 15122432 1028 33554432000 4194304 4785740 140733437817808 18446744073709551615 4553399 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20782/statm: 3692 1028 340 145 0 737 0
[pid=20783] ppid=20780 vsize=456092 CPUtime=1.49 cores=2
/proc/20783/stat : 20783 (TNM) R 20780 20780 20191 0 -1 4202496 448 0 0 0 149 0 0 0 20 0 1 0 493692169 467038208 345 33554432000 134512640 134534564 4290255904 18446744073709551615 134530568 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20783/statm: 114023 345 95 6 0 112689 0
[pid=20784] ppid=20780 vsize=10256 CPUtime=1.49 cores=3
/proc/20784/stat : 20784 (march_hi) R 20780 20780 20191 0 -1 4202496 1317 0 0 0 149 0 0 0 20 0 1 0 493692169 10502144 851 33554432000 4194304 4331988 140733796935344 18446744073709551615 4246336 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20784/statm: 2564 851 175 34 0 1006 0
[pid=20785] ppid=20780 vsize=121816 CPUtime=5.9 cores=4-7
/proc/20785/stat : 20785 (plingeling) S 20780 20780 20191 0 -1 4202496 3404 1257 0 0 590 0 0 0 18 0 5 0 493692169 124739584 2554 33554432000 4194304 4356188 140734736714432 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20785/statm: 30454 2554 163 40 0 28998 0
[pid=20785/tid=20792] ppid=20780 vsize=121816 CPUtime=1.48 cores=4-7
/proc/20785/task/20792/stat : 20792 (plingeling) R 20780 20780 20191 0 -1 4202560 641 1257 0 0 148 0 0 0 23 0 5 0 493692171 124739584 2554 33554432000 4194304 4356188 140734736714432 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20785/tid=20793] ppid=20780 vsize=121816 CPUtime=1.48 cores=4-7
/proc/20785/task/20793/stat : 20793 (plingeling) R 20780 20780 20191 0 -1 4202560 559 1257 0 0 148 0 0 0 23 0 5 0 493692171 124739584 2554 33554432000 4194304 4356188 140734736714432 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20785/tid=20794] ppid=20780 vsize=121816 CPUtime=1.47 cores=4-7
/proc/20785/task/20794/stat : 20794 (plingeling) R 20780 20780 20191 0 -1 4202560 670 1257 0 0 147 0 0 0 23 0 5 0 493692171 124739584 2554 33554432000 4194304 4356188 140734736714432 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20785/tid=20795] ppid=20780 vsize=121816 CPUtime=1.45 cores=4-7
/proc/20785/task/20795/stat : 20795 (plingeling) R 20780 20780 20191 0 -1 4202560 789 1257 0 0 145 0 0 0 23 0 5 0 493692171 124739584 2554 33554432000 4194304 4356188 140734736714432 18446744073709551615 4281757 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.85
Current children cumulated vsize (KiB) 1083112

[startup+3.10103 s]
/proc/loadavg: 8.05 8.04 8.00 9/180 20795
/proc/meminfo: memFree=11210128/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) S 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 4334345759487950847 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
[pid=20781] ppid=20780 vsize=463760 CPUtime=3.08 cores=0
/proc/20781/stat : 20781 (cryptominisat) R 20780 20780 20191 0 -1 4202496 5860 0 0 0 308 0 0 0 25 0 1 0 493692169 474890240 3141 33554432000 4194304 4587354 140736753186480 18446744073709551615 4239308 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20781/statm: 115940 3141 338 96 0 112502 0
[pid=20782] ppid=20780 vsize=15604 CPUtime=3.09 cores=1
/proc/20782/stat : 20782 (clasp) R 20780 20780 20191 0 -1 4202496 1362 0 0 0 309 0 0 0 25 0 1 0 493692169 15978496 1245 33554432000 4194304 4785740 140733437817808 18446744073709551615 4538882 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20782/statm: 3901 1245 340 145 0 946 0
[pid=20783] ppid=20780 vsize=456092 CPUtime=3.09 cores=2
/proc/20783/stat : 20783 (TNM) R 20780 20780 20191 0 -1 4202496 448 0 0 0 309 0 0 0 25 0 1 0 493692169 467038208 345 33554432000 134512640 134534564 4290255904 18446744073709551615 134530669 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20783/statm: 114023 345 95 6 0 112689 0
[pid=20784] ppid=20780 vsize=10424 CPUtime=3.09 cores=3
/proc/20784/stat : 20784 (march_hi) R 20780 20780 20191 0 -1 4202496 1352 0 0 0 309 0 0 0 25 0 1 0 493692169 10674176 886 33554432000 4194304 4331988 140733796935344 18446744073709551615 4228530 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20784/statm: 2606 886 176 34 0 1048 0
[pid=20785] ppid=20780 vsize=124848 CPUtime=12.3 cores=4-7
/proc/20785/stat : 20785 (plingeling) S 20780 20780 20191 0 -1 4202496 4578 1257 0 0 1229 1 0 0 18 0 5 0 493692169 127844352 3312 33554432000 4194304 4356188 140734736714432 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20785/statm: 31212 3312 163 40 0 29756 0
[pid=20785/tid=20792] ppid=20780 vsize=124848 CPUtime=3.08 cores=4-7
/proc/20785/task/20792/stat : 20792 (plingeling) R 20780 20780 20191 0 -1 4202560 963 1257 0 0 308 0 0 0 25 0 5 0 493692171 127844352 3312 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331435 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20785/tid=20793] ppid=20780 vsize=124848 CPUtime=3.07 cores=4-7
/proc/20785/task/20793/stat : 20793 (plingeling) R 20780 20780 20191 0 -1 4202560 832 1257 0 0 307 0 0 0 25 0 5 0 493692171 127844352 3312 33554432000 4194304 4356188 140734736714432 18446744073709551615 4274287 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20785/tid=20794] ppid=20780 vsize=124848 CPUtime=3.07 cores=4-7
/proc/20785/task/20794/stat : 20794 (plingeling) R 20780 20780 20191 0 -1 4202560 864 1257 0 0 307 0 0 0 25 0 5 0 493692171 127844352 3312 33554432000 4194304 4356188 140734736714432 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20785/tid=20795] ppid=20780 vsize=124848 CPUtime=3.04 cores=4-7
/proc/20785/task/20795/stat : 20795 (plingeling) R 20780 20780 20191 0 -1 4202560 1174 1257 0 0 304 0 0 0 25 0 5 0 493692171 127844352 3312 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331309 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.65
Current children cumulated vsize (KiB) 1089052

[startup+6.30046 s]
/proc/loadavg: 8.05 8.04 8.00 9/180 20795
/proc/meminfo: memFree=11200892/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) S 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 4334345759487950847 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
[pid=20781] ppid=20780 vsize=469812 CPUtime=6.29 cores=0
/proc/20781/stat : 20781 (cryptominisat) R 20780 20780 20191 0 -1 4202496 8688 0 0 0 628 1 0 0 25 0 1 0 493692169 481087488 4494 33554432000 4194304 4587354 140736753186480 18446744073709551615 4238600 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20781/statm: 117453 4494 338 96 0 114015 0
[pid=20782] ppid=20780 vsize=16416 CPUtime=6.29 cores=1
/proc/20782/stat : 20782 (clasp) R 20780 20780 20191 0 -1 4202496 1556 0 0 0 629 0 0 0 25 0 1 0 493692169 16809984 1439 33554432000 4194304 4785740 140733437817808 18446744073709551615 4286953 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20782/statm: 4104 1439 340 145 0 1149 0
[pid=20783] ppid=20780 vsize=456092 CPUtime=6.29 cores=2
/proc/20783/stat : 20783 (TNM) R 20780 20780 20191 0 -1 4202496 448 0 0 0 629 0 0 0 25 0 1 0 493692169 467038208 345 33554432000 134512640 134534564 4290255904 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20783/statm: 114023 345 95 6 0 112689 0
[pid=20784] ppid=20780 vsize=10804 CPUtime=6.29 cores=3
/proc/20784/stat : 20784 (march_hi) R 20780 20780 20191 0 -1 4202496 1420 0 0 0 629 0 0 0 25 0 1 0 493692169 11063296 954 33554432000 4194304 4331988 140733796935344 18446744073709551615 4226304 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20784/statm: 2701 954 176 34 0 1143 0
[pid=20785] ppid=20780 vsize=131016 CPUtime=25.08 cores=4-7
/proc/20785/stat : 20785 (plingeling) S 20780 20780 20191 0 -1 4202496 8008 1257 0 0 2507 1 0 0 18 0 5 0 493692169 134160384 4854 33554432000 4194304 4356188 140734736714432 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20785/statm: 32754 4854 163 40 0 31298 0
[pid=20785/tid=20792] ppid=20780 vsize=131016 CPUtime=6.28 cores=4-7
/proc/20785/task/20792/stat : 20792 (plingeling) R 20780 20780 20191 0 -1 4202560 1796 1257 0 0 628 0 0 0 25 0 5 0 493692171 134160384 4854 33554432000 4194304 4356188 140734736714432 18446744073709551615 4330859 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20785/tid=20793] ppid=20780 vsize=131016 CPUtime=6.27 cores=4-7
/proc/20785/task/20793/stat : 20793 (plingeling) R 20780 20780 20191 0 -1 4202560 1922 1257 0 0 627 0 0 0 25 0 5 0 493692171 134160384 4854 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331340 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20785/tid=20794] ppid=20780 vsize=131016 CPUtime=6.27 cores=4-7
/proc/20785/task/20794/stat : 20794 (plingeling) R 20780 20780 20191 0 -1 4202560 1601 1257 0 0 627 0 0 0 25 0 5 0 493692171 134160384 4854 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20785/tid=20795] ppid=20780 vsize=131016 CPUtime=6.22 cores=4-7
/proc/20785/task/20795/stat : 20795 (plingeling) R 20780 20780 20191 0 -1 4202560 1944 1257 0 0 622 0 0 0 25 0 5 0 493692171 134160384 4854 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.24
Current children cumulated vsize (KiB) 1102464

[startup+12.7013 s]
/proc/loadavg: 8.04 8.04 8.00 9/180 20795
/proc/meminfo: memFree=11193036/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) S 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 4334345759487950847 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
[pid=20781] ppid=20780 vsize=468824 CPUtime=12.69 cores=0
/proc/20781/stat : 20781 (cryptominisat) R 20780 20780 20191 0 -1 4202496 12958 0 0 0 1267 2 0 0 25 0 1 0 493692169 480075776 4280 33554432000 4194304 4587354 140736753186480 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20781/statm: 117206 4280 338 96 0 113768 0
[pid=20782] ppid=20780 vsize=17528 CPUtime=12.69 cores=1
/proc/20782/stat : 20782 (clasp) R 20780 20780 20191 0 -1 4202496 1819 0 0 0 1269 0 0 0 25 0 1 0 493692169 17948672 1702 33554432000 4194304 4785740 140733437817808 18446744073709551615 4552558 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20782/statm: 4382 1702 341 145 0 1427 0
[pid=20783] ppid=20780 vsize=456092 CPUtime=12.69 cores=2
/proc/20783/stat : 20783 (TNM) R 20780 20780 20191 0 -1 4202496 448 0 0 0 1269 0 0 0 25 0 1 0 493692169 467038208 345 33554432000 134512640 134534564 4290255904 18446744073709551615 134530806 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20783/statm: 114023 345 95 6 0 112689 0
[pid=20784] ppid=20780 vsize=11804 CPUtime=12.69 cores=3
/proc/20784/stat : 20784 (march_hi) R 20780 20780 20191 0 -1 4202496 1588 0 0 0 1269 0 0 0 25 0 1 0 493692169 12087296 1122 33554432000 4194304 4331988 140733796935344 18446744073709551615 4237416 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20784/statm: 2951 1122 176 34 0 1393 0
[pid=20785] ppid=20780 vsize=130092 CPUtime=50.64 cores=4-7
/proc/20785/stat : 20785 (plingeling) S 20780 20780 20191 0 -1 4202496 10633 1257 0 0 5063 1 0 0 18 0 5 0 493692169 133214208 4623 33554432000 4194304 4356188 140734736714432 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20785/statm: 32523 4623 163 40 0 31067 0
[pid=20785/tid=20792] ppid=20780 vsize=130092 CPUtime=12.68 cores=4-7
/proc/20785/task/20792/stat : 20792 (plingeling) R 20780 20780 20191 0 -1 4202560 2532 1257 0 0 1268 0 0 0 25 0 5 0 493692171 133214208 4623 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20785/tid=20793] ppid=20780 vsize=130092 CPUtime=12.67 cores=4-7
/proc/20785/task/20793/stat : 20793 (plingeling) R 20780 20780 20191 0 -1 4202560 2787 1257 0 0 1267 0 0 0 25 0 5 0 493692171 133214208 4623 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20785/tid=20794] ppid=20780 vsize=130092 CPUtime=12.67 cores=4-7
/proc/20785/task/20794/stat : 20794 (plingeling) R 20780 20780 20191 0 -1 4202560 1985 1257 0 0 1267 0 0 0 25 0 5 0 493692171 133214208 4623 33554432000 4194304 4356188 140734736714432 18446744073709551615 4330973 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20785/tid=20795] ppid=20780 vsize=130092 CPUtime=12.59 cores=4-7
/proc/20785/task/20795/stat : 20795 (plingeling) R 20780 20780 20191 0 -1 4202560 2584 1257 0 0 1259 0 0 0 25 0 5 0 493692171 133214208 4623 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.4
Current children cumulated vsize (KiB) 1102664

[startup+25.501 s]
/proc/loadavg: 8.03 8.04 8.00 9/180 20796
/proc/meminfo: memFree=11183908/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) S 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 4334345759487950847 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
[pid=20781] ppid=20780 vsize=466604 CPUtime=25.49 cores=0
/proc/20781/stat : 20781 (cryptominisat) R 20780 20780 20191 0 -1 4202496 20531 0 0 0 2545 4 0 0 25 0 1 0 493692169 477802496 5822 33554432000 4194304 4587354 140736753186480 18446744073709551615 4238608 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=20783] ppid=20780 vsize=456096 CPUtime=4782.4 cores=2
/proc/20783/stat : 20783 (TNM) R 20780 20780 20191 0 -1 4202496 457 0 0 0 478239 1 0 0 25 0 1 0 493692169 467042304 354 33554432000 134512640 134534564 4290255904 18446744073709551615 8440833 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20783/statm: 114024 354 103 6 0 112690 0
[pid=20784] ppid=20780 vsize=495256 CPUtime=4781.6 cores=3
/proc/20784/stat : 20784 (march_hi) R 20780 20780 20191 0 -1 4202496 97143 0 0 0 478094 66 0 0 25 0 1 0 493692169 507142144 96677 33554432000 4194304 4331988 140733796935344 18446744073709551615 4287489 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20784/statm: 123814 96677 176 34 0 122256 0
[pid=20785] ppid=20780 vsize=241744 CPUtime=19103.1 cores=4-7
/proc/20785/stat : 20785 (plingeling) S 20780 20780 20191 0 -1 4202496 522074 1257 0 0 1910223 85 0 0 18 0 5 0 493692169 247545856 32536 33554432000 4194304 4356188 140734736714432 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20785/statm: 60436 32536 163 40 0 58980 0
[pid=20785/tid=20792] ppid=20780 vsize=241744 CPUtime=4782.22 cores=4-7
/proc/20785/task/20792/stat : 20792 (plingeling) R 20780 20780 20191 0 -1 4202560 142134 1257 0 0 478201 21 0 0 25 0 5 0 493692171 247545856 32536 33554432000 4194304 4356188 140734736714432 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20785/tid=20793] ppid=20780 vsize=241744 CPUtime=4782.39 cores=4-7
/proc/20785/task/20793/stat : 20793 (plingeling) R 20780 20780 20191 0 -1 4202560 116808 1257 0 0 478218 21 0 0 25 0 5 0 493692171 247545856 32536 33554432000 4194304 4356188 140734736714432 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20785/tid=20794] ppid=20780 vsize=241744 CPUtime=4782.1 cores=4-7
/proc/20785/task/20794/stat : 20794 (plingeling) R 20780 20780 20191 0 -1 4202560 116133 1257 0 0 478190 20 0 0 25 0 5 0 493692171 247545856 32536 33554432000 4194304 4356188 140734736714432 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20785/tid=20795] ppid=20780 vsize=241744 CPUtime=4756.32 cores=4-7
/proc/20785/task/20795/stat : 20795 (plingeling) R 20780 20780 20191 0 -1 4202560 146254 1257 0 0 475611 21 0 0 25 0 5 0 493692171 247545856 32536 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38231.8
Current children cumulated vsize (KiB) 7755160

[startup+4842.31 s]
/proc/loadavg: 8.06 8.04 8.00 9/180 20945
/proc/meminfo: memFree=10402048/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) S 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 4334345759487950847 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
[pid=20781] ppid=20780 vsize=6486508 CPUtime=4842.28 cores=0
/proc/20781/stat : 20781 (cryptominisat) R 20780 20780 20191 0 -1 4202496 524973 0 0 0 484016 212 0 0 25 0 1 0 493692169 6642184192 70802 33554432000 4194304 4587354 140736753186480 18446744073709551615 4357662 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20781/statm: 1621627 70802 338 96 0 1618189 0
[pid=20782] ppid=20780 vsize=57232 CPUtime=4842.41 cores=1
/proc/20782/stat : 20782 (clasp) R 20780 20780 20191 0 -1 4202496 10848 0 0 0 484240 1 0 0 25 0 1 0 493692169 58605568 10731 33554432000 4194304 4785740 140733437817808 18446744073709551615 4287104 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20782/statm: 14308 10731 341 145 0 11353 0
[pid=20783] ppid=20780 vsize=456096 CPUtime=4842.41 cores=2
/proc/20783/stat : 20783 (TNM) R 20780 20780 20191 0 -1 4202496 457 0 0 0 484240 1 0 0 25 0 1 0 493692169 467042304 354 33554432000 134512640 134534564 4290255904 18446744073709551615 134530533 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20783/statm: 114024 354 103 6 0 112690 0
[pid=20784] ppid=20780 vsize=495256 CPUtime=4841.61 cores=3
/proc/20784/stat : 20784 (march_hi) R 20780 20780 20191 0 -1 4202496 98300 0 0 0 484093 68 0 0 25 0 1 0 493692169 507142144 97834 33554432000 4194304 4331988 140733796935344 18446744073709551615 4228534 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20784/statm: 123814 97834 176 34 0 122256 0
[pid=20785] ppid=20780 vsize=234320 CPUtime=19342.8 cores=4-7
/proc/20785/stat : 20785 (plingeling) S 20780 20780 20191 0 -1 4202496 524314 1257 0 0 1934192 86 0 0 18 0 5 0 493692169 239943680 30680 33554432000 4194304 4356188 140734736714432 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20785/statm: 58580 30680 163 40 0 57124 0
[pid=20785/tid=20792] ppid=20780 vsize=234320 CPUtime=4842.23 cores=4-7
/proc/20785/task/20792/stat : 20792 (plingeling) R 20780 20780 20191 0 -1 4202560 142806 1257 0 0 484201 22 0 0 25 0 5 0 493692171 239943680 30680 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20785/tid=20793] ppid=20780 vsize=234320 CPUtime=4842.4 cores=4-7
/proc/20785/task/20793/stat : 20793 (plingeling) R 20780 20780 20191 0 -1 4202560 117096 1257 0 0 484219 21 0 0 25 0 5 0 493692171 239943680 30680 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20785/tid=20794] ppid=20780 vsize=234320 CPUtime=4842.1 cores=4-7
/proc/20785/task/20794/stat : 20794 (plingeling) R 20780 20780 20191 0 -1 4202560 116133 1257 0 0 484190 20 0 0 25 0 5 0 493692171 239943680 30680 33554432000 4194304 4356188 140734736714432 18446744073709551615 4279769 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20785/tid=20795] ppid=20780 vsize=234320 CPUtime=4816 cores=4-7
/proc/20785/task/20795/stat : 20795 (plingeling) R 20780 20780 20191 0 -1 4202560 147534 1257 0 0 481579 21 0 0 25 0 5 0 493692171 239943680 30680 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331964 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38711.5
Current children cumulated vsize (KiB) 7747736

[startup+4902.3 s]
/proc/loadavg: 8.06 8.04 8.00 9/180 20947
/proc/meminfo: memFree=10371760/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) S 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 4334345759487950847 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
[pid=20781] ppid=20780 vsize=6486508 CPUtime=4902.27 cores=0
/proc/20781/stat : 20781 (cryptominisat) R 20780 20780 20191 0 -1 4202496 524973 0 0 0 490015 212 0 0 25 0 1 0 493692169 6642184192 70802 33554432000 4194304 4587354 140736753186480 18446744073709551615 4357646 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20781/statm: 1621627 70802 338 96 0 1618189 0
[pid=20782] ppid=20780 vsize=57416 CPUtime=4902.41 cores=1
/proc/20782/stat : 20782 (clasp) R 20780 20780 20191 0 -1 4202496 10861 0 0 0 490240 1 0 0 25 0 1 0 493692169 58793984 10744 33554432000 4194304 4785740 140733437817808 18446744073709551615 4553375 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20782/statm: 14354 10744 341 145 0 11399 0
[pid=20783] ppid=20780 vsize=456096 CPUtime=4902.4 cores=2
/proc/20783/stat : 20783 (TNM) R 20780 20780 20191 0 -1 4202496 457 0 0 0 490239 1 0 0 25 0 1 0 493692169 467042304 354 33554432000 134512640 134534564 4290255904 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20783/statm: 114024 354 103 6 0 112690 0
[pid=20784] ppid=20780 vsize=495256 CPUtime=4901.61 cores=3
/proc/20784/stat : 20784 (march_hi) R 20780 20780 20191 0 -1 4202496 99474 0 0 0 490092 69 0 0 25 0 1 0 493692169 507142144 99008 33554432000 4194304 4331988 140733796935344 18446744073709551615 4308710 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20784/statm: 123814 99008 176 34 0 122256 0
[pid=20785] ppid=20780 vsize=259536 CPUtime=19582.4 cores=4-7
/proc/20785/stat : 20785 (plingeling) S 20780 20780 20191 0 -1 4202496 532474 1257 0 0 1958157 87 0 0 18 0 5 0 493692169 265764864 36984 33554432000 4194304 4356188 140734736714432 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20785/statm: 64884 36984 163 40 0 63428 0
[pid=20785/tid=20792] ppid=20780 vsize=259536 CPUtime=4902.22 cores=4-7
/proc/20785/task/20792/stat : 20792 (plingeling) R 20780 20780 20191 0 -1 4202560 143830 1257 0 0 490200 22 0 0 25 0 5 0 493692171 265764864 36984 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20785/tid=20793] ppid=20780 vsize=259536 CPUtime=4902.4 cores=4-7
/proc/20785/task/20793/stat : 20793 (plingeling) R 20780 20780 20191 0 -1 4202560 120200 1257 0 0 490219 21 0 0 25 0 5 0 493692171 265764864 36984 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20785/tid=20794] ppid=20780 vsize=259536 CPUtime=4902.1 cores=4-7
/proc/20785/task/20794/stat : 20794 (plingeling) R 20780 20780 20191 0 -1 4202560 116261 1257 0 0 490190 20 0 0 25 0 5 0 493692171 265764864 36984 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20785/tid=20795] ppid=20780 vsize=259536 CPUtime=4875.68 cores=4-7
/proc/20785/task/20795/stat : 20795 (plingeling) R 20780 20780 20191 0 -1 4202560 151438 1257 0 0 487546 22 0 0 25 0 5 0 493692171 265764864 36984 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39191.1
Current children cumulated vsize (KiB) 7773136

[startup+4962.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/180 20948
/proc/meminfo: memFree=10378920/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) S 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 4334345759487950847 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
[pid=20781] ppid=20780 vsize=6486508 CPUtime=4962.27 cores=0
/proc/20781/stat : 20781 (cryptominisat) R 20780 20780 20191 0 -1 4202496 524973 0 0 0 496015 212 0 0 25 0 1 0 493692169 6642184192 70802 33554432000 4194304 4587354 140736753186480 18446744073709551615 4357642 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20781/statm: 1621627 70802 338 96 0 1618189 0
[pid=20782] ppid=20780 vsize=57808 CPUtime=4962.41 cores=1
/proc/20782/stat : 20782 (clasp) R 20780 20780 20191 0 -1 4202496 10944 0 0 0 496240 1 0 0 25 0 1 0 493692169 59195392 10827 33554432000 4194304 4785740 140733437817808 18446744073709551615 4546099 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20782/statm: 14452 10827 341 145 0 11497 0
[pid=20783] ppid=20780 vsize=456096 CPUtime=4962.41 cores=2
/proc/20783/stat : 20783 (TNM) R 20780 20780 20191 0 -1 4202496 457 0 0 0 496240 1 0 0 25 0 1 0 493692169 467042304 354 33554432000 134512640 134534564 4290255904 18446744073709551615 134530820 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20783/statm: 114024 354 103 6 0 112690 0
[pid=20784] ppid=20780 vsize=495256 CPUtime=4961.61 cores=3
/proc/20784/stat : 20784 (march_hi) R 20780 20780 20191 0 -1 4202496 100631 0 0 0 496091 70 0 0 25 0 1 0 493692169 507142144 100165 33554432000 4194304 4331988 140733796935344 18446744073709551615 4299342 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20784/statm: 123814 100165 176 34 0 122256 0
[pid=20785] ppid=20780 vsize=247120 CPUtime=19822.1 cores=4-7
/proc/20785/stat : 20785 (plingeling) S 20780 20780 20191 0 -1 4202496 533658 1257 0 0 1982124 87 0 0 18 0 5 0 493692169 253050880 33880 33554432000 4194304 4356188 140734736714432 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20785/statm: 61780 33880 163 40 0 60324 0
[pid=20785/tid=20792] ppid=20780 vsize=247120 CPUtime=4962.22 cores=4-7
/proc/20785/task/20792/stat : 20792 (plingeling) R 20780 20780 20191 0 -1 4202560 144246 1257 0 0 496200 22 0 0 25 0 5 0 493692171 253050880 33880 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331007 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20785/tid=20793] ppid=20780 vsize=247120 CPUtime=4962.4 cores=4-7
/proc/20785/task/20793/stat : 20793 (plingeling) R 20780 20780 20191 0 -1 4202560 120584 1257 0 0 496219 21 0 0 25 0 5 0 493692171 253050880 33880 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331173 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20785/tid=20794] ppid=20780 vsize=247120 CPUtime=4962.1 cores=4-7
/proc/20785/task/20794/stat : 20794 (plingeling) R 20780 20780 20191 0 -1 4202560 116453 1257 0 0 496190 20 0 0 25 0 5 0 493692171 253050880 33880 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20785/tid=20795] ppid=20780 vsize=247120 CPUtime=4935.35 cores=4-7
/proc/20785/task/20795/stat : 20795 (plingeling) R 20780 20780 20191 0 -1 4202560 151630 1257 0 0 493513 22 0 0 25 0 5 0 493692171 253050880 33880 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331062 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39670.8
Current children cumulated vsize (KiB) 7761112



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.5 s]
/proc/loadavg: 8.08 8.05 8.00 9/180 20949
/proc/meminfo: memFree=10368852/32951124 swapFree=67111528/67111528
[pid=20780] ppid=20778 vsize=18324 CPUtime=0 cores=0-7
/proc/20780/stat : 20780 (ppfolio) S 20778 20780 20191 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 493692169 18763776 389 33554432000 4194304 4287625 140735802360432 18446744073709551615 233510317583 0 0 4096 65536 4334345759487950847 0 0 17 6 0 0 0
/proc/20780/statm: 4581 389 341 23 0 91 0
[pid=20781] ppid=20780 vsize=6486508 CPUtime=5003.48 cores=0
/proc/20781/stat : 20781 (cryptominisat) R 20780 20780 20191 0 -1 4202496 524973 0 0 0 500136 212 0 0 25 0 1 0 493692169 6642184192 70802 33554432000 4194304 4587354 140736753186480 18446744073709551615 4357604 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20781/statm: 1621627 70802 338 96 0 1618189 0
[pid=20782] ppid=20780 vsize=57808 CPUtime=5003.62 cores=1
/proc/20782/stat : 20782 (clasp) R 20780 20780 20191 0 -1 4202496 10953 0 0 0 500361 1 0 0 25 0 1 0 493692169 59195392 10836 33554432000 4194304 4785740 140733437817808 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/20782/statm: 14452 10836 341 145 0 11497 0
[pid=20783] ppid=20780 vsize=456096 CPUtime=5003.61 cores=2
/proc/20783/stat : 20783 (TNM) R 20780 20780 20191 0 -1 4202496 457 0 0 0 500360 1 0 0 25 0 1 0 493692169 467042304 354 33554432000 134512640 134534564 4290255904 18446744073709551615 8440832 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/20783/statm: 114024 354 103 6 0 112690 0
[pid=20784] ppid=20780 vsize=495256 CPUtime=5002.82 cores=3
/proc/20784/stat : 20784 (march_hi) R 20780 20780 20191 0 -1 4202496 101419 0 0 0 500211 71 0 0 25 0 1 0 493692169 507142144 100953 33554432000 4194304 4331988 140733796935344 18446744073709551615 4242538 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/20784/statm: 123814 100953 176 34 0 122256 0
[pid=20785] ppid=20780 vsize=254544 CPUtime=19986.7 cores=4-7
/proc/20785/stat : 20785 (plingeling) S 20780 20780 20191 0 -1 4202496 537562 1257 0 0 1998582 87 0 0 18 0 5 0 493692169 260653056 35736 33554432000 4194304 4356188 140734736714432 18446744073709551615 233522100197 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/20785/statm: 63636 35736 163 40 0 62180 0
[pid=20785/tid=20792] ppid=20780 vsize=254544 CPUtime=5003.42 cores=4-7
/proc/20785/task/20792/stat : 20792 (plingeling) R 20780 20780 20191 0 -1 4202560 147446 1257 0 0 500320 22 0 0 25 0 5 0 493692171 260653056 35736 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20785/tid=20793] ppid=20780 vsize=254544 CPUtime=5003.6 cores=4-7
/proc/20785/task/20793/stat : 20793 (plingeling) R 20780 20780 20191 0 -1 4202560 120584 1257 0 0 500339 21 0 0 25 0 5 0 493692171 260653056 35736 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331028 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=20785/tid=20794] ppid=20780 vsize=254544 CPUtime=5003.31 cores=4-7
/proc/20785/task/20794/stat : 20794 (plingeling) R 20780 20780 20191 0 -1 4202560 116581 1257 0 0 500311 20 0 0 25 0 5 0 493692171 260653056 35736 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=20785/tid=20795] ppid=20780 vsize=254544 CPUtime=4976.33 cores=4-7
/proc/20785/task/20795/stat : 20795 (plingeling) R 20780 20780 20191 0 -1 4202560 152206 1257 0 0 497611 22 0 0 25 0 5 0 493692171 260653056 35736 33554432000 4194304 4356188 140734736714432 18446744073709551615 4331264 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 7768536

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 20780 and gives
#  childrusage.ru_utime.tv_sec=10007
#  childrusage.ru_utime.tv_usec=220671
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=43993
# CPU time returned by wait4() is 10007.3
# while last known CPU time is 40000.2
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5003.52
CPU time (s): 40000.2
CPU user time (s): 39996.5
CPU system time (s): 3.72
CPU usage (%): 799.442
Max. virtual memory (cumulated for all children) (KiB): 13499444

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 10007.2
system time used= 0.043993
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 11946
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 631
involuntary context switches= 26764

runsolver used 8.02478 second user time and 19.2421 second system time

The end

Launcher Data 

Begin job on node145 at 2011-04-28 19:46:03
IDJOB=3310331
IDBENCH=83418
IDSOLVER=1638
FILE ID=node145/3310331-1304012763
RUNJOBID= node145-1304007357-20205
PBS_JOBID= 13168830
Free space on /tmp= 72208 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/kullmann/AES/Benchmarks/64/aes_64_2_keyfind_1.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3310331-1304012763/watcher-3310331-1304012763 -o /tmp/evaluation-result-3310331-1304012763/solver-3310331-1304012763 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3310331-1304012763.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 1024a3346322fdba0a20b51f99eae4c0
RANDOM SEED=818872230

node145.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      11238884 kB
Buffers:       2202244 kB
Cached:       18864176 kB
SwapCached:          0 kB
Active:       10925824 kB
Inactive:     10209396 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      11238884 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           10096 kB
Writeback:           0 kB
AnonPages:       68408 kB
Mapped:          14708 kB
Slab:           509820 kB
PageTables:       4640 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   187072 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72196 MiB
End job on node145 at 2011-04-28 21:09:29