Trace number 3310325

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 35670.3 5100.02

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom23.cnf
MD5SUMe560aa6505ddce6d3784f70d5b7aff66
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables57800
Number of clauses193923
Sum of the clauses size495331
Maximum clause length1803
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 298497
Number of clauses of size 394437
Number of clauses of size 4275
Number of clauses of size 521
Number of clauses of size over 5692

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3310325-1304011676/watcher-3310325-1304011676 -o /tmp/evaluation-result-3310325-1304011676/solver-3310325-1304011676 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3310325-1304011676.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.36 7.89 7.96 1/170 30688
/proc/meminfo: memFree=20048624/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 512767341 18763776 390 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 15182866625817739263 0 0 17 4 0 0 0
/proc/30688/statm: 4581 390 341 23 0 91 0

[startup+0.021803 s]
/proc/loadavg: 7.36 7.89 7.96 1/170 30688
/proc/meminfo: memFree=20048624/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 512767341 18763776 390 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 15182866625817739263 0 0 17 4 0 0 0
/proc/30688/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100787 s]
/proc/loadavg: 7.36 7.89 7.96 1/170 30688
/proc/meminfo: memFree=20048624/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 25 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195738567704576 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.300757 s]
/proc/loadavg: 7.36 7.89 7.96 1/170 30688
/proc/meminfo: memFree=20048624/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 25 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195738567704576 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.700688 s]
/proc/loadavg: 7.36 7.89 7.96 1/170 30688
/proc/meminfo: memFree=20048624/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 25 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195738567704576 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+1.5006 s]
/proc/loadavg: 7.36 7.89 7.96 8/179 30703
/proc/meminfo: memFree=19883260/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 15 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195738567704576 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
[pid=30689] ppid=30688 vsize=70916 CPUtime=1.48 cores=0
/proc/30689/stat : 30689 (cryptominisat) R 30688 30688 30662 0 -1 4202496 14980 0 0 0 147 1 0 0 20 0 1 0 512767341 72617984 12744 33554432000 4194304 4587354 140735147337920 18446744073709551615 4420280 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30689/statm: 17729 12744 317 96 0 14291 0
[pid=30690] ppid=30688 vsize=46188 CPUtime=1.48 cores=1
/proc/30690/stat : 30690 (clasp) R 30688 30688 30662 0 -1 4202496 11161 0 0 0 147 1 0 0 20 0 1 0 512767341 47296512 8691 33554432000 4194304 4785740 140735171357232 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30690/statm: 11547 8691 340 145 0 8592 0
[pid=30692] ppid=30688 vsize=126460 CPUtime=1.49 cores=3
/proc/30692/stat : 30692 (march_hi) R 30688 30688 30662 0 -1 4202496 74927 0 0 0 139 10 0 0 20 0 1 0 512767341 129495040 15018 33554432000 4194304 4331988 140737097170752 18446744073709551615 4263314 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30692/statm: 31615 15018 224 34 0 30057 0
[pid=30693] ppid=30688 vsize=101720 CPUtime=3.98 cores=4-7
/proc/30693/stat : 30693 (plingeling) S 30688 30688 30662 0 -1 4202496 22698 1248 0 0 396 2 0 0 19 0 5 0 512767341 104161280 13851 33554432000 4194304 4356188 140734723733888 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/30693/statm: 25430 13851 159 40 0 23974 0
[pid=30693/tid=30700] ppid=30688 vsize=101720 CPUtime=0.63 cores=4-7
/proc/30693/task/30700/stat : 30700 (plingeling) R 30688 30688 30662 0 -1 4202560 3386 1248 0 0 63 0 0 0 22 0 5 0 512767365 104161280 13851 33554432000 4194304 4356188 140734723733888 18446744073709551615 4315974 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=30693/tid=30701] ppid=30688 vsize=101720 CPUtime=0.61 cores=4-7
/proc/30693/task/30701/stat : 30701 (plingeling) R 30688 30688 30662 0 -1 4202560 2553 1248 0 0 61 0 0 0 22 0 5 0 512767365 104161280 13851 33554432000 4194304 4356188 140734723733888 18446744073709551615 4322077 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=30693/tid=30702] ppid=30688 vsize=101720 CPUtime=1.23 cores=4-7
/proc/30693/task/30702/stat : 30702 (plingeling) R 30688 30688 30662 0 -1 4202560 3512 1248 0 0 123 0 0 0 25 0 5 0 512767365 104161280 13851 33554432000 4194304 4356188 140734723733888 18446744073709551615 4322000 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30693/tid=30703] ppid=30688 vsize=101720 CPUtime=1.25 cores=4-7
/proc/30693/task/30703/stat : 30703 (plingeling) R 30688 30688 30662 0 -1 4202560 4607 1248 0 0 125 0 0 0 25 0 5 0 512767365 104161280 13851 33554432000 4194304 4356188 140734723733888 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 8.44
Current children cumulated vsize (KiB) 363608

[startup+3.10126 s]
/proc/loadavg: 7.36 7.89 7.96 8/179 30703
/proc/meminfo: memFree=19834796/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 15 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195742862639364 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
[pid=30689] ppid=30688 vsize=114276 CPUtime=3.09 cores=0
/proc/30689/stat : 30689 (cryptominisat) R 30688 30688 30662 0 -1 4202496 18634 0 0 0 307 2 0 0 25 0 1 0 512767341 117018624 14680 33554432000 4194304 4587354 140735147337920 18446744073709551615 4238608 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30689/statm: 28569 14680 330 96 0 25131 0
[pid=30690] ppid=30688 vsize=46320 CPUtime=3.08 cores=1
/proc/30690/stat : 30690 (clasp) R 30688 30688 30662 0 -1 4202496 11235 0 0 0 307 1 0 0 25 0 1 0 512767341 47431680 8765 33554432000 4194304 4785740 140735171357232 18446744073709551615 4553375 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30690/statm: 11580 8765 340 145 0 8625 0
[pid=30692] ppid=30688 vsize=134100 CPUtime=3.09 cores=3
/proc/30692/stat : 30692 (march_hi) R 30688 30688 30662 0 -1 4202496 77229 0 0 0 299 10 0 0 25 0 1 0 512767341 137318400 17265 33554432000 4194304 4331988 140737097170752 18446744073709551615 4288315 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30692/statm: 33525 17265 176 34 0 31967 0
[pid=30693] ppid=30688 vsize=110108 CPUtime=9.85 cores=4-7
/proc/30693/stat : 30693 (plingeling) S 30688 30688 30662 0 -1 4202496 24817 1248 0 0 983 2 0 0 19 0 5 0 512767341 112750592 15970 33554432000 4194304 4356188 140734723733888 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/30693/statm: 27527 15970 163 40 0 26071 0
[pid=30693/tid=30700] ppid=30688 vsize=110108 CPUtime=1.99 cores=4-7
/proc/30693/task/30700/stat : 30700 (plingeling) R 30688 30688 30662 0 -1 4202560 3648 1248 0 0 199 0 0 0 25 0 5 0 512767365 112750592 15970 33554432000 4194304 4356188 140734723733888 18446744073709551615 4217785 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30693/tid=30701] ppid=30688 vsize=110108 CPUtime=1.92 cores=4-7
/proc/30693/task/30701/stat : 30701 (plingeling) R 30688 30688 30662 0 -1 4202560 3455 1248 0 0 192 0 0 0 25 0 5 0 512767365 112750592 15970 33554432000 4194304 4356188 140734723733888 18446744073709551615 4302050 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=30693/tid=30702] ppid=30688 vsize=110108 CPUtime=2.83 cores=4-7
/proc/30693/task/30702/stat : 30702 (plingeling) R 30688 30688 30662 0 -1 4202560 4080 1248 0 0 283 0 0 0 25 0 5 0 512767365 112750592 15970 33554432000 4194304 4356188 140734723733888 18446744073709551615 4216975 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30693/tid=30703] ppid=30688 vsize=110108 CPUtime=2.85 cores=4-7
/proc/30693/task/30703/stat : 30703 (plingeling) R 30688 30688 30662 0 -1 4202560 4994 1248 0 0 285 0 0 0 25 0 5 0 512767365 112750592 15970 33554432000 4194304 4356188 140734723733888 18446744073709551615 4241556 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 19.12
Current children cumulated vsize (KiB) 423128

[startup+6.30069 s]
/proc/loadavg: 7.33 7.88 7.95 8/179 30703
/proc/meminfo: memFree=19809224/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 15 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195742862639364 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
[pid=30689] ppid=30688 vsize=1831192 CPUtime=6.28 cores=0
/proc/30689/stat : 30689 (cryptominisat) R 30688 30688 30662 0 -1 4202496 21370 0 0 0 626 2 0 0 25 0 1 0 512767341 1875140608 14321 33554432000 4194304 4587354 140735147337920 18446744073709551615 4230333 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30689/statm: 457798 14321 333 96 0 454360 0
[pid=30690] ppid=30688 vsize=51580 CPUtime=6.29 cores=1
/proc/30690/stat : 30690 (clasp) R 30688 30688 30662 0 -1 4202496 12540 0 0 0 627 2 0 0 25 0 1 0 512767341 52817920 10070 33554432000 4194304 4785740 140735171357232 18446744073709551615 237278540485 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30690/statm: 12895 10070 340 145 0 9940 0
[pid=30692] ppid=30688 vsize=138452 CPUtime=6.28 cores=3
/proc/30692/stat : 30692 (march_hi) R 30688 30688 30662 0 -1 4202496 78486 0 0 0 618 10 0 0 25 0 1 0 512767341 141774848 18521 33554432000 4194304 4331988 140737097170752 18446744073709551615 4237136 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30692/statm: 34613 18521 175 34 0 33055 0
[pid=30693] ppid=30688 vsize=115272 CPUtime=22.62 cores=4-7
/proc/30693/stat : 30693 (plingeling) S 30688 30688 30662 0 -1 4202496 29277 1248 0 0 2260 2 0 0 19 0 5 0 512767341 118038528 17261 33554432000 4194304 4356188 140734723733888 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/30693/statm: 28818 17261 163 40 0 27362 0
[pid=30693/tid=30700] ppid=30688 vsize=115272 CPUtime=5.17 cores=4-7
/proc/30693/task/30700/stat : 30700 (plingeling) R 30688 30688 30662 0 -1 4202560 4672 1248 0 0 517 0 0 0 25 0 5 0 512767365 118038528 17261 33554432000 4194304 4356188 140734723733888 18446744073709551615 4332098 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30693/tid=30701] ppid=30688 vsize=115272 CPUtime=5.11 cores=4-7
/proc/30693/task/30701/stat : 30701 (plingeling) R 30688 30688 30662 0 -1 4202560 4479 1248 0 0 511 0 0 0 25 0 5 0 512767365 118038528 17261 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=30693/tid=30702] ppid=30688 vsize=115272 CPUtime=6.03 cores=4-7
/proc/30693/task/30702/stat : 30702 (plingeling) R 30688 30688 30662 0 -1 4202560 5104 1248 0 0 603 0 0 0 25 0 5 0 512767365 118038528 17261 33554432000 4194304 4356188 140734723733888 18446744073709551615 4330995 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30693/tid=30703] ppid=30688 vsize=115272 CPUtime=6.05 cores=4-7
/proc/30693/task/30703/stat : 30703 (plingeling) R 30688 30688 30662 0 -1 4202560 6382 1248 0 0 605 0 0 0 25 0 5 0 512767365 118038528 17261 33554432000 4194304 4356188 140734723733888 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 41.48
Current children cumulated vsize (KiB) 2154820

[startup+12.7006 s]
/proc/loadavg: 7.30 7.86 7.95 8/179 30703
/proc/meminfo: memFree=19773428/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 15 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195742862639364 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
[pid=30689] ppid=30688 vsize=1843268 CPUtime=12.68 cores=0
/proc/30689/stat : 30689 (cryptominisat) R 30688 30688 30662 0 -1 4202496 25847 0 0 0 1265 3 0 0 25 0 1 0 512767341 1887506432 18780 33554432000 4194304 4587354 140735147337920 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30689/statm: 460817 18780 336 96 0 457379 0
[pid=30690] ppid=30688 vsize=62992 CPUtime=12.69 cores=1
/proc/30690/stat : 30690 (clasp) R 30688 30688 30662 0 -1 4202496 15379 0 0 0 1266 3 0 0 25 0 1 0 512767341 64503808 12909 33554432000 4194304 4785740 140735171357232 18446744073709551615 4552628 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30690/statm: 15748 12909 340 145 0 12793 0
[pid=30692] ppid=30688 vsize=145632 CPUtime=12.69 cores=3
/proc/30692/stat : 30692 (march_hi) R 30688 30688 30662 0 -1 4202496 80015 0 0 0 1258 11 0 0 25 0 1 0 512767341 149127168 20050 33554432000 4194304 4331988 140737097170752 18446744073709551615 4237139 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30692/statm: 36408 20050 175 34 0 34850 0
[pid=30693] ppid=30688 vsize=115232 CPUtime=48.19 cores=4-7
/proc/30693/stat : 30693 (plingeling) S 30688 30688 30662 0 -1 4202496 29277 1248 0 0 4817 2 0 0 19 0 5 0 512767341 117997568 17251 33554432000 4194304 4356188 140734723733888 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/30693/statm: 28808 17251 163 40 0 27352 0
[pid=30693/tid=30700] ppid=30688 vsize=115232 CPUtime=11.54 cores=4-7
/proc/30693/task/30700/stat : 30700 (plingeling) R 30688 30688 30662 0 -1 4202560 4672 1248 0 0 1154 0 0 0 25 0 5 0 512767365 117997568 17251 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30693/tid=30701] ppid=30688 vsize=115232 CPUtime=11.51 cores=4-7
/proc/30693/task/30701/stat : 30701 (plingeling) R 30688 30688 30662 0 -1 4202560 4479 1248 0 0 1151 0 0 0 25 0 5 0 512767365 117997568 17251 33554432000 4194304 4356188 140734723733888 18446744073709551615 4279717 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=30693/tid=30702] ppid=30688 vsize=115232 CPUtime=12.43 cores=4-7
/proc/30693/task/30702/stat : 30702 (plingeling) R 30688 30688 30662 0 -1 4202560 5104 1248 0 0 1243 0 0 0 25 0 5 0 512767365 117997568 17251 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331050 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30693/tid=30703] ppid=30688 vsize=115232 CPUtime=12.45 cores=4-7
/proc/30693/task/30703/stat : 30703 (plingeling) R 30688 30688 30662 0 -1 4202560 6382 1248 0 0 1245 0 0 0 25 0 5 0 512767365 117997568 17251 33554432000 4194304 4356188 140734723733888 18446744073709551615 4211015 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 86.26
Current children cumulated vsize (KiB) 2185448

[startup+25.5013 s]
/proc/loadavg: 7.31 7.83 7.94 8/179 30704
/proc/meminfo: memFree=19756244/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 15 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195742862639364 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
[pid=30689] ppid=30688 vsize=103960 CPUtime=25.49 cores=0
/proc/30689/stat : 30689 (cryptominisat) R 30688 30688 30662 0 -1 4202496 33725 0 0 0 2544 5 0 0 25 0 1 0 512767341 106455040 20512 33554432000 4194304 4587354 140735147337920 18446744073709551615 4238755 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30689/statm: 25990 20512 336 96 0 22552 0
[pid=30690] ppid=30688 vsize=71780 CPUtime=25.48 cores=1
/proc/30690/stat : 30690 (clasp) R 30688 30688 30662 0 -1 4202496 17512 0 0 0 2545 3 0 0 25 0 1 0 512767341 73502720 15042 33554432000 4194304 4785740 140735171357232 18446744073709551615 4552628 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30690/statm: 17945 15042 340 145 0 14990 0
[pid=30692] ppid=30688 vsize=147664 CPUtime=25.49 cores=3
/proc/30692/stat : 30692 (march_hi) R 30688 30688 30662 0 -1 4202496 80577 0 0 0 2538 11 0 0 25 0 1 0 512767341 151207936 20612 33554432000 4194304 4331988 140737097170752 18446744073709551615 4263143 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30692/statm: 36916 20612 175 34 0 35358 0
[pid=30693] ppid=30688 vsize=180752 CPUtime=99.32 cores=4-7
/proc/30693/stat : 30693 (plingeling) S 30688 30688 30662 0 -1 4202496 29313 1248 0 0 9930 2 0 0 19 0 5 0 512767341 185090048 17280 33554432000 4194304 4356188 140734723733888 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/30693/statm: 45188 17280 163 40 0 43732 0
[pid=30693/tid=30700] ppid=30688 vsize=180752 CPUtime=24.27 cores=4-7
/proc/30693/task/30700/stat : 30700 (plingeling) R 30688 30688 30662 0 -1 4202560 4672 1248 0 0 2427 0 0 0 25 0 5 0 512767365 185090048 17280 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331062 0 0 4096 17442 0 0 0 -1 7 0 0 0

################
# More data... #
################

Current children cumulated CPU time (s) 33867.8
Current children cumulated vsize (KiB) 2909008

[startup+4902.3 s]
/proc/loadavg: 7.00 7.00 7.00 8/179 30846
/proc/meminfo: memFree=19094012/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 15 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195742862639364 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
[pid=30689] ppid=30688 vsize=1905936 CPUtime=4902.41 cores=0
/proc/30689/stat : 30689 (cryptominisat) R 30688 30688 30662 0 -1 4202496 998332 0 0 0 489877 364 0 0 25 0 1 0 512767341 1951678464 41695 33554432000 4194304 4587354 140735147337920 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30689/statm: 476484 41695 336 96 0 473046 0
[pid=30690] ppid=30688 vsize=433136 CPUtime=4902.26 cores=1
/proc/30690/stat : 30690 (clasp) R 30688 30688 30662 0 -1 4202496 105751 0 0 0 490176 50 0 0 25 0 1 0 512767341 443531264 103281 33554432000 4194304 4785740 140735171357232 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30690/statm: 108284 103281 340 145 0 105329 0
[pid=30692] ppid=30688 vsize=185872 CPUtime=4902.41 cores=3
/proc/30692/stat : 30692 (march_hi) R 30688 30688 30662 0 -1 4202496 90180 0 0 0 490223 18 0 0 25 0 1 0 512767341 190332928 30215 33554432000 4194304 4331988 140737097170752 18446744073709551615 4237136 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30692/statm: 46468 30215 175 34 0 44910 0
[pid=30693] ppid=30688 vsize=365740 CPUtime=19580.4 cores=4-7
/proc/30693/stat : 30693 (plingeling) S 30688 30688 30662 0 -1 4202496 350034 1248 0 0 1957989 48 0 0 19 0 5 0 512767341 374517760 63528 33554432000 4194304 4356188 140734723733888 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/30693/statm: 91435 63528 164 40 0 89979 0
[pid=30693/tid=30700] ppid=30688 vsize=365740 CPUtime=4874.59 cores=4-7
/proc/30693/task/30700/stat : 30700 (plingeling) R 30688 30688 30662 0 -1 4202560 91983 1248 0 0 487446 13 0 0 25 0 5 0 512767365 374517760 63528 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30693/tid=30701] ppid=30688 vsize=365740 CPUtime=4901.23 cores=4-7
/proc/30693/task/30701/stat : 30701 (plingeling) R 30688 30688 30662 0 -1 4202560 123415 1248 0 0 490108 15 0 0 25 0 5 0 512767365 374517760 63528 33554432000 4194304 4356188 140734723733888 18446744073709551615 4210817 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=30693/tid=30702] ppid=30688 vsize=365740 CPUtime=4902.16 cores=4-7
/proc/30693/task/30702/stat : 30702 (plingeling) R 30688 30688 30662 0 -1 4202560 87555 1248 0 0 490203 13 0 0 25 0 5 0 512767365 374517760 63528 33554432000 4194304 4356188 140734723733888 18446744073709551615 4280568 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30693/tid=30703] ppid=30688 vsize=365740 CPUtime=4902.12 cores=4-7
/proc/30693/task/30703/stat : 30703 (plingeling) R 30688 30688 30662 0 -1 4202560 38441 1248 0 0 490208 4 0 0 25 0 5 0 512767365 374517760 63528 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 34287.5
Current children cumulated vsize (KiB) 2909008

[startup+4962.3 s]
/proc/loadavg: 7.00 7.00 7.00 8/179 30847
/proc/meminfo: memFree=19074040/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 15 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195738567704576 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
[pid=30689] ppid=30688 vsize=1906040 CPUtime=4962.41 cores=0
/proc/30689/stat : 30689 (cryptominisat) R 30688 30688 30662 0 -1 4202496 1023046 0 0 0 495872 369 0 0 25 0 1 0 512767341 1951784960 38312 33554432000 4194304 4587354 140735147337920 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30689/statm: 476510 38312 336 96 0 473072 0
[pid=30690] ppid=30688 vsize=433136 CPUtime=4962.26 cores=1
/proc/30690/stat : 30690 (clasp) R 30688 30688 30662 0 -1 4202496 105751 0 0 0 496176 50 0 0 25 0 1 0 512767341 443531264 103281 33554432000 4194304 4785740 140735171357232 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30690/statm: 108284 103281 340 145 0 105329 0
[pid=30692] ppid=30688 vsize=185872 CPUtime=4962.41 cores=3
/proc/30692/stat : 30692 (march_hi) R 30688 30688 30662 0 -1 4202496 90244 0 0 0 496223 18 0 0 25 0 1 0 512767341 190332928 30279 33554432000 4194304 4331988 140737097170752 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30692/statm: 46468 30279 175 34 0 44910 0
[pid=30693] ppid=30688 vsize=398508 CPUtime=19820.1 cores=4-7
/proc/30693/stat : 30693 (plingeling) S 30688 30688 30662 0 -1 4202496 359762 1248 0 0 1981956 50 0 0 19 0 5 0 512767341 408072192 71720 33554432000 4194304 4356188 140734723733888 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/30693/statm: 99627 71720 164 40 0 98171 0
[pid=30693/tid=30700] ppid=30688 vsize=398508 CPUtime=4934.27 cores=4-7
/proc/30693/task/30700/stat : 30700 (plingeling) R 30688 30688 30662 0 -1 4202560 96079 1248 0 0 493413 14 0 0 25 0 5 0 512767365 408072192 71720 33554432000 4194304 4356188 140734723733888 18446744073709551615 4284076 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30693/tid=30701] ppid=30688 vsize=398508 CPUtime=4961.24 cores=4-7
/proc/30693/task/30701/stat : 30701 (plingeling) R 30688 30688 30662 0 -1 4202560 123415 1248 0 0 496109 15 0 0 25 0 5 0 512767365 408072192 71720 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331392 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=30693/tid=30702] ppid=30688 vsize=398508 CPUtime=4962.16 cores=4-7
/proc/30693/task/30702/stat : 30702 (plingeling) R 30688 30688 30662 0 -1 4202560 92675 1248 0 0 496202 14 0 0 25 0 5 0 512767365 408072192 71720 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30693/tid=30703] ppid=30688 vsize=398508 CPUtime=4962.12 cores=4-7
/proc/30693/task/30703/stat : 30703 (plingeling) R 30688 30688 30662 0 -1 4202560 38953 1248 0 0 496208 4 0 0 25 0 5 0 512767365 408072192 71720 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 34707.2
Current children cumulated vsize (KiB) 2941880

[startup+5022.3 s]
/proc/loadavg: 7.00 7.00 7.00 8/179 30849
/proc/meminfo: memFree=19112348/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 15 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195738567704576 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
[pid=30689] ppid=30688 vsize=1925120 CPUtime=5022.42 cores=0
/proc/30689/stat : 30689 (cryptominisat) R 30688 30688 30662 0 -1 4202496 1034889 0 0 0 501868 374 0 0 25 0 1 0 512767341 1971322880 31729 33554432000 4194304 4587354 140735147337920 18446744073709551615 4238525 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30689/statm: 481280 31729 336 96 0 477842 0
[pid=30690] ppid=30688 vsize=433136 CPUtime=5022.26 cores=1
/proc/30690/stat : 30690 (clasp) R 30688 30688 30662 0 -1 4202496 105751 0 0 0 502176 50 0 0 25 0 1 0 512767341 443531264 103281 33554432000 4194304 4785740 140735171357232 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30690/statm: 108284 103281 340 145 0 105329 0
[pid=30692] ppid=30688 vsize=194736 CPUtime=5022.42 cores=3
/proc/30692/stat : 30692 (march_hi) R 30688 30688 30662 0 -1 4202496 90313 0 0 0 502224 18 0 0 25 0 1 0 512767341 199409664 30348 33554432000 4194304 4331988 140737097170752 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30692/statm: 48684 30348 175 34 0 47126 0
[pid=30693] ppid=30688 vsize=386220 CPUtime=20059.7 cores=4-7
/proc/30693/stat : 30693 (plingeling) S 30688 30688 30662 0 -1 4202496 362322 1248 0 0 2005923 50 0 0 19 0 5 0 512767341 395489280 68648 33554432000 4194304 4356188 140734723733888 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/30693/statm: 96555 68648 164 40 0 95099 0
[pid=30693/tid=30700] ppid=30688 vsize=386220 CPUtime=4993.94 cores=4-7
/proc/30693/task/30700/stat : 30700 (plingeling) R 30688 30688 30662 0 -1 4202560 98127 1248 0 0 499380 14 0 0 25 0 5 0 512767365 395489280 68648 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30693/tid=30701] ppid=30688 vsize=386220 CPUtime=5021.24 cores=4-7
/proc/30693/task/30701/stat : 30701 (plingeling) R 30688 30688 30662 0 -1 4202560 123415 1248 0 0 502109 15 0 0 25 0 5 0 512767365 395489280 68648 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=30693/tid=30702] ppid=30688 vsize=386220 CPUtime=5022.16 cores=4-7
/proc/30693/task/30702/stat : 30702 (plingeling) R 30688 30688 30662 0 -1 4202560 92675 1248 0 0 502202 14 0 0 25 0 5 0 512767365 395489280 68648 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30693/tid=30703] ppid=30688 vsize=386220 CPUtime=5022.12 cores=4-7
/proc/30693/task/30703/stat : 30703 (plingeling) R 30688 30688 30662 0 -1 4202560 39465 1248 0 0 502208 4 0 0 25 0 5 0 512767365 395489280 68648 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331407 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 35126.8
Current children cumulated vsize (KiB) 2957536

[startup+5082.3 s]
/proc/loadavg: 7.05 7.01 7.00 8/179 30851
/proc/meminfo: memFree=19096348/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 15 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195738567704576 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
[pid=30689] ppid=30688 vsize=1925120 CPUtime=5082.41 cores=0
/proc/30689/stat : 30689 (cryptominisat) R 30688 30688 30662 0 -1 4202496 1042997 0 0 0 507864 377 0 0 25 0 1 0 512767341 1971322880 39750 33554432000 4194304 4587354 140735147337920 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30689/statm: 481280 39750 336 96 0 477842 0
[pid=30690] ppid=30688 vsize=433136 CPUtime=5082.26 cores=1
/proc/30690/stat : 30690 (clasp) R 30688 30688 30662 0 -1 4202496 105764 0 0 0 508176 50 0 0 25 0 1 0 512767341 443531264 103294 33554432000 4194304 4785740 140735171357232 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30690/statm: 108284 103294 340 145 0 105329 0
[pid=30692] ppid=30688 vsize=194736 CPUtime=5082.42 cores=3
/proc/30692/stat : 30692 (march_hi) R 30688 30688 30662 0 -1 4202496 90380 0 0 0 508224 18 0 0 25 0 1 0 512767341 199409664 30415 33554432000 4194304 4331988 140737097170752 18446744073709551615 237278274625 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30692/statm: 48684 30415 175 34 0 47126 0
[pid=30693] ppid=30688 vsize=369836 CPUtime=20299.4 cores=4-7
/proc/30693/stat : 30693 (plingeling) S 30688 30688 30662 0 -1 4202496 363858 1248 0 0 2029890 51 0 0 19 0 5 0 512767341 378712064 64552 33554432000 4194304 4356188 140734723733888 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/30693/statm: 92459 64552 164 40 0 91003 0
[pid=30693/tid=30700] ppid=30688 vsize=369836 CPUtime=5053.61 cores=4-7
/proc/30693/task/30700/stat : 30700 (plingeling) R 30688 30688 30662 0 -1 4202560 98127 1248 0 0 505347 14 0 0 25 0 5 0 512767365 378712064 64552 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30693/tid=30701] ppid=30688 vsize=369836 CPUtime=5081.25 cores=4-7
/proc/30693/task/30701/stat : 30701 (plingeling) R 30688 30688 30662 0 -1 4202560 124439 1248 0 0 508109 16 0 0 25 0 5 0 512767365 378712064 64552 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331383 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=30693/tid=30702] ppid=30688 vsize=369836 CPUtime=5082.16 cores=4-7
/proc/30693/task/30702/stat : 30702 (plingeling) R 30688 30688 30662 0 -1 4202560 92675 1248 0 0 508202 14 0 0 25 0 5 0 512767365 378712064 64552 33554432000 4194304 4356188 140734723733888 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30693/tid=30703] ppid=30688 vsize=369836 CPUtime=5082.12 cores=4-7
/proc/30693/task/30703/stat : 30703 (plingeling) R 30688 30688 30662 0 -1 4202560 39977 1248 0 0 508208 4 0 0 25 0 5 0 512767365 378712064 64552 33554432000 4194304 4356188 140734723733888 18446744073709551615 4273866 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 35546.5
Current children cumulated vsize (KiB) 2941152



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 7.04 7.01 7.00 8/179 30851
/proc/meminfo: memFree=19088284/32950904 swapFree=67082284/67111528
[pid=30688] ppid=30686 vsize=18324 CPUtime=0.01 cores=0-7
/proc/30688/stat : 30688 (ppfolio) S 30686 30688 30662 0 -1 4202496 524 1194 0 0 0 0 0 1 15 0 1 0 512767341 18763776 391 33554432000 4194304 4287625 140734322505664 18446744073709551615 237278899727 0 0 4096 65536 9224195738567704576 0 0 17 4 0 0 0
/proc/30688/statm: 4581 391 342 23 0 91 0
[pid=30689] ppid=30688 vsize=1925120 CPUtime=5100.11 cores=0
/proc/30689/stat : 30689 (cryptominisat) R 30688 30688 30662 0 -1 4202496 1044661 0 0 0 509633 378 0 0 25 0 1 0 512767341 1971322880 41381 33554432000 4194304 4587354 140735147337920 18446744073709551615 4238692 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/30689/statm: 481280 41381 336 96 0 477842 0
[pid=30690] ppid=30688 vsize=433136 CPUtime=5099.96 cores=1
/proc/30690/stat : 30690 (clasp) R 30688 30688 30662 0 -1 4202496 105764 0 0 0 509946 50 0 0 25 0 1 0 512767341 443531264 103294 33554432000 4194304 4785740 140735171357232 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/30690/statm: 108284 103294 340 145 0 105329 0
[pid=30692] ppid=30688 vsize=194736 CPUtime=5100.12 cores=3
/proc/30692/stat : 30692 (march_hi) R 30688 30688 30662 0 -1 4202496 90399 0 0 0 509994 18 0 0 25 0 1 0 512767341 199409664 30434 33554432000 4194304 4331988 140737097170752 18446744073709551615 4265729 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/30692/statm: 48684 30434 175 34 0 47126 0
[pid=30693] ppid=30688 vsize=371884 CPUtime=20370.1 cores=4-7
/proc/30693/stat : 30693 (plingeling) S 30688 30688 30662 0 -1 4202496 364370 1248 0 0 2036963 51 0 0 19 0 5 0 512767341 380809216 65064 33554432000 4194304 4356188 140734723733888 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/30693/statm: 92971 65064 164 40 0 91515 0
[pid=30693/tid=30700] ppid=30688 vsize=371884 CPUtime=5071.22 cores=4-7
/proc/30693/task/30700/stat : 30700 (plingeling) R 30688 30688 30662 0 -1 4202560 98127 1248 0 0 507108 14 0 0 25 0 5 0 512767365 380809216 65064 33554432000 4194304 4356188 140734723733888 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=30693/tid=30701] ppid=30688 vsize=371884 CPUtime=5098.95 cores=4-7
/proc/30693/task/30701/stat : 30701 (plingeling) R 30688 30688 30662 0 -1 4202560 124439 1248 0 0 509879 16 0 0 25 0 5 0 512767365 380809216 65064 33554432000 4194304 4356188 140734723733888 18446744073709551615 4281690 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=30693/tid=30702] ppid=30688 vsize=371884 CPUtime=5099.87 cores=4-7
/proc/30693/task/30702/stat : 30702 (plingeling) R 30688 30688 30662 0 -1 4202560 93187 1248 0 0 509973 14 0 0 25 0 5 0 512767365 380809216 65064 33554432000 4194304 4356188 140734723733888 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=30693/tid=30703] ppid=30688 vsize=371884 CPUtime=5099.82 cores=4-7
/proc/30693/task/30703/stat : 30703 (plingeling) R 30688 30688 30662 0 -1 4202560 39977 1248 0 0 509978 4 0 0 25 0 5 0 512767365 380809216 65064 33554432000 4194304 4356188 140734723733888 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 35670.3
Current children cumulated vsize (KiB) 2943200

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 30688 and gives
#  childrusage.ru_utime.tv_sec=10196
#  childrusage.ru_utime.tv_usec=293927
#  childrusage.ru_stime.tv_sec=4
#  childrusage.ru_stime.tv_usec=12390
# CPU time returned by wait4() is 10200.3
# while last known CPU time is 35670.3
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5100.02
CPU time (s): 35670.3
CPU user time (s): 35665.4
CPU system time (s): 4.98
CPU usage (%): 699.416
Max. virtual memory (cumulated for all children) (KiB): 4947632

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 10196.3
system time used= 4.01239
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1136778
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 583
involuntary context switches= 68372

runsolver used 8.40172 second user time and 19.425 second system time

The end

Launcher Data 

Begin job on node121 at 2011-04-28 19:27:56
IDJOB=3310325
IDBENCH=83137
IDSOLVER=1638
FILE ID=node121/3310325-1304011676
RUNJOBID= node121-1304011676-30676
PBS_JOBID= 13168857
Free space on /tmp= 72652 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom23.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3310325-1304011676/watcher-3310325-1304011676 -o /tmp/evaluation-result-3310325-1304011676/solver-3310325-1304011676 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3310325-1304011676.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= e560aa6505ddce6d3784f70d5b7aff66
RANDOM SEED=634797657

node121.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950904 kB
MemFree:      20049028 kB
Buffers:       2176544 kB
Cached:       10166328 kB
SwapCached:       2776 kB
Active:        9090596 kB
Inactive:      3290436 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950904 kB
LowFree:      20049028 kB
SwapTotal:    67111528 kB
SwapFree:     67082284 kB
Dirty:           13408 kB
Writeback:           0 kB
AnonPages:       36672 kB
Mapped:          13368 kB
Slab:           454848 kB
PageTables:       4632 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586980 kB
Committed_AS:   183672 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72636 MiB
End job on node121 at 2011-04-28 20:52:58