Trace number 3221847

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39971.3 5000.02

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v600-c2556-S506401817-075.UNKNOWN.cnf
MD5SUM2d7407b1b64ab354711b83851f166439
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables600
Number of clauses2556
Sum of the clauses size7668
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32556
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3221847-1304162815/watcher-3221847-1304162815 -o /tmp/evaluation-result-3221847-1304162815/solver-3221847-1304162815 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3221847-1304162815.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.06 8.04 8.01 1/170 4602
/proc/meminfo: memFree=24810924/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) S 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 10666635808867352575 0 0 17 4 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0

[startup+0.0332329 s]
/proc/loadavg: 8.06 8.04 8.01 1/170 4602
/proc/meminfo: memFree=24810924/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) R 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988592384 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100221 s]
/proc/loadavg: 8.06 8.04 8.01 1/170 4602
/proc/meminfo: memFree=24810924/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) R 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300186 s]
/proc/loadavg: 8.06 8.04 8.01 1/170 4602
/proc/meminfo: memFree=24810924/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) R 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 24 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700116 s]
/proc/loadavg: 8.06 8.04 8.01 1/170 4602
/proc/meminfo: memFree=24810924/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) R 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 22 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50098 s]
/proc/loadavg: 8.06 8.04 8.01 9/180 4617
/proc/meminfo: memFree=24792972/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) S 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 10666635808867352575 0 0 17 7 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
[pid=4603] ppid=4602 vsize=713452 CPUtime=1.49 cores=0
/proc/4603/stat : 4603 (cryptominisat) R 4602 4602 3977 0 -1 4202496 3938 0 0 0 149 0 0 0 25 0 1 0 606512304 730574848 2642 33554432000 4194304 4587354 140735640025072 18446744073709551615 4238668 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4603/statm: 178363 2642 337 96 0 174925 0
[pid=4604] ppid=4602 vsize=14296 CPUtime=1.49 cores=1
/proc/4604/stat : 4604 (clasp) R 4602 4602 3977 0 -1 4202496 1029 0 0 0 149 0 0 0 20 0 1 0 606512304 14639104 916 33554432000 4194304 4785740 140733420045872 18446744073709551615 4287008 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4604/statm: 3574 916 340 145 0 619 0
[pid=4605] ppid=4602 vsize=455960 CPUtime=1.49 cores=2
/proc/4605/stat : 4605 (TNM) R 4602 4602 3977 0 -1 4202496 356 0 0 0 149 0 0 0 25 0 1 0 606512304 466903040 253 33554432000 134512640 134534564 4287689392 18446744073709551615 134528730 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4605/statm: 113990 253 95 6 0 112656 0
[pid=4606] ppid=4602 vsize=8256 CPUtime=1.49 cores=3
/proc/4606/stat : 4606 (march_hi) R 4602 4602 3977 0 -1 4202496 670 0 0 0 149 0 0 0 20 0 1 0 606512304 8454144 563 33554432000 4194304 4331988 140734796245120 18446744073709551615 4211493 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4606/statm: 2064 563 173 34 0 506 0
[pid=4607] ppid=4602 vsize=122844 CPUtime=5.91 cores=4-7
/proc/4607/stat : 4607 (plingeling) S 4602 4602 3977 0 -1 4202496 3661 1250 0 0 591 0 0 0 19 0 5 0 606512304 125792256 2830 33554432000 4194304 4356188 140735583514224 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4607/statm: 30711 2830 164 40 0 29255 0
[pid=4607/tid=4614] ppid=4602 vsize=122844 CPUtime=1.48 cores=4-7
/proc/4607/task/4614/stat : 4614 (plingeling) R 4602 4602 3977 0 -1 4202560 788 1250 0 0 148 0 0 0 20 0 5 0 606512305 125792256 2830 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4607/tid=4615] ppid=4602 vsize=122844 CPUtime=1.48 cores=4-7
/proc/4607/task/4615/stat : 4615 (plingeling) R 4602 4602 3977 0 -1 4202560 756 1250 0 0 148 0 0 0 23 0 5 0 606512305 125792256 2830 33554432000 4194304 4356188 140735583514224 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4607/tid=4616] ppid=4602 vsize=122844 CPUtime=1.46 cores=4-7
/proc/4607/task/4616/stat : 4616 (plingeling) R 4602 4602 3977 0 -1 4202560 723 1250 0 0 146 0 0 0 25 0 5 0 606512305 125792256 2830 33554432000 4194304 4356188 140735583514224 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4607/tid=4617] ppid=4602 vsize=122844 CPUtime=1.47 cores=4-7
/proc/4607/task/4617/stat : 4617 (plingeling) R 4602 4602 3977 0 -1 4202560 861 1250 0 0 147 0 0 0 25 0 5 0 606512305 125792256 2830 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331389 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.87
Current children cumulated vsize (KiB) 1333132

[startup+3.1007 s]
/proc/loadavg: 8.06 8.04 8.01 9/180 4617
/proc/meminfo: memFree=24785408/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) S 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 10666635808867352575 0 0 17 7 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
[pid=4603] ppid=4602 vsize=717224 CPUtime=3.09 cores=0
/proc/4603/stat : 4603 (cryptominisat) R 4602 4602 3977 0 -1 4202496 5960 0 0 0 309 0 0 0 25 0 1 0 606512304 734437376 3750 33554432000 4194304 4587354 140735640025072 18446744073709551615 4364768 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4603/statm: 179306 3750 337 96 0 175868 0
[pid=4604] ppid=4602 vsize=15100 CPUtime=3.09 cores=1
/proc/4604/stat : 4604 (clasp) R 4602 4602 3977 0 -1 4202496 1230 0 0 0 309 0 0 0 25 0 1 0 606512304 15462400 1117 33554432000 4194304 4785740 140733420045872 18446744073709551615 4416832 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4604/statm: 3775 1117 340 145 0 820 0
[pid=4605] ppid=4602 vsize=455960 CPUtime=3.09 cores=2
/proc/4605/stat : 4605 (TNM) R 4602 4602 3977 0 -1 4202496 356 0 0 0 309 0 0 0 25 0 1 0 606512304 466903040 253 33554432000 134512640 134534564 4287689392 18446744073709551615 134529812 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4605/statm: 113990 253 95 6 0 112656 0
[pid=4606] ppid=4602 vsize=8636 CPUtime=3.09 cores=3
/proc/4606/stat : 4606 (march_hi) R 4602 4602 3977 0 -1 4202496 729 0 0 0 309 0 0 0 25 0 1 0 606512304 8843264 622 33554432000 4194304 4331988 140734796245120 18446744073709551615 4240519 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4606/statm: 2159 622 173 34 0 601 0
[pid=4607] ppid=4602 vsize=123120 CPUtime=12.3 cores=4-7
/proc/4607/stat : 4607 (plingeling) S 4602 4602 3977 0 -1 4202496 5014 1250 0 0 1230 0 0 0 19 0 5 0 606512304 126074880 2903 33554432000 4194304 4356188 140735583514224 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4607/statm: 30780 2903 164 40 0 29324 0
[pid=4607/tid=4614] ppid=4602 vsize=123120 CPUtime=3.08 cores=4-7
/proc/4607/task/4614/stat : 4614 (plingeling) R 4602 4602 3977 0 -1 4202560 1142 1250 0 0 308 0 0 0 25 0 5 0 606512305 126074880 2903 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331328 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4607/tid=4615] ppid=4602 vsize=123120 CPUtime=3.08 cores=4-7
/proc/4607/task/4615/stat : 4615 (plingeling) R 4602 4602 3977 0 -1 4202560 1145 1250 0 0 308 0 0 0 25 0 5 0 606512305 126074880 2903 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4607/tid=4616] ppid=4602 vsize=123120 CPUtime=3.05 cores=4-7
/proc/4607/task/4616/stat : 4616 (plingeling) R 4602 4602 3977 0 -1 4202560 1044 1250 0 0 305 0 0 0 25 0 5 0 606512305 126074880 2903 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331165 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4607/tid=4617] ppid=4602 vsize=123120 CPUtime=3.07 cores=4-7
/proc/4607/task/4617/stat : 4617 (plingeling) R 4602 4602 3977 0 -1 4202560 1150 1250 0 0 307 0 0 0 25 0 5 0 606512305 126074880 2903 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.66
Current children cumulated vsize (KiB) 1338364

[startup+6.30114 s]
/proc/loadavg: 8.06 8.03 8.00 9/180 4617
/proc/meminfo: memFree=24773132/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) S 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 10666635808867352575 0 0 17 7 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
[pid=4603] ppid=4602 vsize=717268 CPUtime=6.29 cores=0
/proc/4603/stat : 4603 (cryptominisat) R 4602 4602 3977 0 -1 4202496 7948 0 0 0 629 0 0 0 25 0 1 0 606512304 734482432 3988 33554432000 4194304 4587354 140735640025072 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4603/statm: 179317 3988 337 96 0 175879 0
[pid=4604] ppid=4602 vsize=16320 CPUtime=6.29 cores=1
/proc/4604/stat : 4604 (clasp) R 4602 4602 3977 0 -1 4202496 1547 0 0 0 629 0 0 0 25 0 1 0 606512304 16711680 1434 33554432000 4194304 4785740 140733420045872 18446744073709551615 4553032 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4604/statm: 4080 1434 340 145 0 1125 0
[pid=4605] ppid=4602 vsize=455960 CPUtime=6.29 cores=2
/proc/4605/stat : 4605 (TNM) R 4602 4602 3977 0 -1 4202496 356 0 0 0 629 0 0 0 25 0 1 0 606512304 466903040 253 33554432000 134512640 134534564 4287689392 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4605/statm: 113990 253 95 6 0 112656 0
[pid=4606] ppid=4602 vsize=9112 CPUtime=6.29 cores=3
/proc/4606/stat : 4606 (march_hi) R 4602 4602 3977 0 -1 4202496 872 0 0 0 629 0 0 0 25 0 1 0 606512304 9330688 765 33554432000 4194304 4331988 140734796245120 18446744073709551615 215987987472 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4606/statm: 2278 765 173 34 0 720 0
[pid=4607] ppid=4602 vsize=124536 CPUtime=25.1 cores=4-7
/proc/4607/stat : 4607 (plingeling) S 4602 4602 3977 0 -1 4202496 7699 1250 0 0 2509 1 0 0 19 0 5 0 606512304 127524864 3257 33554432000 4194304 4356188 140735583514224 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4607/statm: 31134 3257 164 40 0 29678 0
[pid=4607/tid=4614] ppid=4602 vsize=124536 CPUtime=6.28 cores=4-7
/proc/4607/task/4614/stat : 4614 (plingeling) R 4602 4602 3977 0 -1 4202560 1780 1250 0 0 628 0 0 0 25 0 5 0 606512305 127524864 3257 33554432000 4194304 4356188 140735583514224 18446744073709551615 4330817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4607/tid=4615] ppid=4602 vsize=124536 CPUtime=6.28 cores=4-7
/proc/4607/task/4615/stat : 4615 (plingeling) R 4602 4602 3977 0 -1 4202560 1848 1250 0 0 628 0 0 0 25 0 5 0 606512305 127524864 3257 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331345 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4607/tid=4616] ppid=4602 vsize=124536 CPUtime=6.25 cores=4-7
/proc/4607/task/4616/stat : 4616 (plingeling) R 4602 4602 3977 0 -1 4202560 1716 1250 0 0 625 0 0 0 25 0 5 0 606512305 127524864 3257 33554432000 4194304 4356188 140735583514224 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4607/tid=4617] ppid=4602 vsize=124536 CPUtime=6.25 cores=4-7
/proc/4607/task/4617/stat : 4617 (plingeling) R 4602 4602 3977 0 -1 4202560 1822 1250 0 0 625 0 0 0 25 0 5 0 606512305 127524864 3257 33554432000 4194304 4356188 140735583514224 18446744073709551615 4234306 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.26
Current children cumulated vsize (KiB) 1341520

[startup+12.701 s]
/proc/loadavg: 8.05 8.03 8.00 9/180 4617
/proc/meminfo: memFree=24761908/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) S 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 10666635808867352575 0 0 17 7 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
[pid=4603] ppid=4602 vsize=57168 CPUtime=12.68 cores=0
/proc/4603/stat : 4603 (cryptominisat) R 4602 4602 3977 0 -1 4202496 10101 0 0 0 1268 0 0 0 25 0 1 0 606512304 58540032 5408 33554432000 4194304 4587354 140735640025072 18446744073709551615 4238731 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4603/statm: 14292 5408 337 96 0 10854 0
[pid=4604] ppid=4602 vsize=17948 CPUtime=12.69 cores=1
/proc/4604/stat : 4604 (clasp) R 4602 4602 3977 0 -1 4202496 1932 0 0 0 1269 0 0 0 25 0 1 0 606512304 18378752 1819 33554432000 4194304 4785740 140733420045872 18446744073709551615 4552514 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4604/statm: 4487 1819 340 145 0 1532 0
[pid=4605] ppid=4602 vsize=455960 CPUtime=12.69 cores=2
/proc/4605/stat : 4605 (TNM) R 4602 4602 3977 0 -1 4202496 356 0 0 0 1269 0 0 0 25 0 1 0 606512304 466903040 253 33554432000 134512640 134534564 4287689392 18446744073709551615 134530487 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4605/statm: 113990 253 95 6 0 112656 0
[pid=4606] ppid=4602 vsize=10524 CPUtime=12.69 cores=3
/proc/4606/stat : 4606 (march_hi) R 4602 4602 3977 0 -1 4202496 1115 0 0 0 1269 0 0 0 25 0 1 0 606512304 10776576 1008 33554432000 4194304 4331988 140734796245120 18446744073709551615 215988289028 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4606/statm: 2631 1008 173 34 0 1073 0
[pid=4607] ppid=4602 vsize=130440 CPUtime=50.65 cores=4-7
/proc/4607/stat : 4607 (plingeling) S 4602 4602 3977 0 -1 4202496 13591 1250 0 0 5064 1 0 0 19 0 5 0 606512304 133570560 4733 33554432000 4194304 4356188 140735583514224 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4607/statm: 32610 4733 164 40 0 31154 0
[pid=4607/tid=4614] ppid=4602 vsize=130440 CPUtime=12.68 cores=4-7
/proc/4607/task/4614/stat : 4614 (plingeling) R 4602 4602 3977 0 -1 4202560 3221 1250 0 0 1268 0 0 0 25 0 5 0 606512305 133570560 4733 33554432000 4194304 4356188 140735583514224 18446744073709551615 4332125 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4607/tid=4615] ppid=4602 vsize=130440 CPUtime=12.68 cores=4-7
/proc/4607/task/4615/stat : 4615 (plingeling) R 4602 4602 3977 0 -1 4202560 3449 1250 0 0 1268 0 0 0 25 0 5 0 606512305 133570560 4733 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331416 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4607/tid=4616] ppid=4602 vsize=130440 CPUtime=12.65 cores=4-7
/proc/4607/task/4616/stat : 4616 (plingeling) R 4602 4602 3977 0 -1 4202560 3125 1250 0 0 1265 0 0 0 25 0 5 0 606512305 133570560 4733 33554432000 4194304 4356188 140735583514224 18446744073709551615 4281766 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4607/tid=4617] ppid=4602 vsize=130440 CPUtime=12.61 cores=4-7
/proc/4607/task/4617/stat : 4617 (plingeling) R 4602 4602 3977 0 -1 4202560 3263 1250 0 0 1261 0 0 0 25 0 5 0 606512305 133570560 4733 33554432000 4194304 4356188 140735583514224 18446744073709551615 4274287 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.4
Current children cumulated vsize (KiB) 690364

[startup+25.5008 s]
/proc/loadavg: 8.04 8.03 8.00 9/180 4618
/proc/meminfo: memFree=24757256/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) S 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 10666635808867352575 0 0 17 7 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
[pid=4603] ppid=4602 vsize=721168 CPUtime=25.49 cores=0
/proc/4603/stat : 4603 (cryptominisat) R 4602 4602 3977 0 -1 4202496 12745 0 0 0 2548 1 0 0 25 0 1 0 606512304 738476032 4064 33554432000 4194304 4587354 140735640025072 18446744073709551615 4238568 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=4605] ppid=4602 vsize=455964 CPUtime=4782.28 cores=2
/proc/4605/stat : 4605 (TNM) R 4602 4602 3977 0 -1 4202496 365 0 0 0 478227 1 0 0 25 0 1 0 606512304 466907136 262 33554432000 134512640 134534564 4287689392 18446744073709551615 134529880 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4605/statm: 113991 262 103 6 0 112657 0
[pid=4606] ppid=4602 vsize=596028 CPUtime=4782.39 cores=3
/proc/4606/stat : 4606 (march_hi) R 4602 4602 3977 0 -1 4202496 132776 0 0 0 478126 113 0 0 25 0 1 0 606512304 610332672 132669 33554432000 4194304 4331988 140734796245120 18446744073709551615 4211461 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4606/statm: 149007 132669 173 34 0 147449 0
[pid=4607] ppid=4602 vsize=271564 CPUtime=19101.4 cores=4-7
/proc/4607/stat : 4607 (plingeling) S 4602 4602 3977 0 -1 4202496 973768 1250 0 0 1909985 155 0 0 19 0 5 0 606512304 278081536 40010 33554432000 4194304 4356188 140735583514224 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4607/statm: 67891 40010 164 40 0 66435 0
[pid=4607/tid=4614] ppid=4602 vsize=271564 CPUtime=4782.39 cores=4-7
/proc/4607/task/4614/stat : 4614 (plingeling) R 4602 4602 3977 0 -1 4202560 257402 1250 0 0 478194 45 0 0 25 0 5 0 606512305 278081536 40010 33554432000 4194304 4356188 140735583514224 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4607/tid=4615] ppid=4602 vsize=271564 CPUtime=4781.92 cores=4-7
/proc/4607/task/4615/stat : 4615 (plingeling) R 4602 4602 3977 0 -1 4202560 231773 1250 0 0 478158 34 0 0 25 0 5 0 606512305 278081536 40010 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4607/tid=4616] ppid=4602 vsize=271564 CPUtime=4782.36 cores=4-7
/proc/4607/task/4616/stat : 4616 (plingeling) R 4602 4602 3977 0 -1 4202560 241016 1250 0 0 478201 35 0 0 25 0 5 0 606512305 278081536 40010 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331050 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4607/tid=4617] ppid=4602 vsize=271564 CPUtime=4754.69 cores=4-7
/proc/4607/task/4617/stat : 4617 (plingeling) R 4602 4602 3977 0 -1 4202560 243044 1250 0 0 475430 39 0 0 25 0 5 0 606512305 278081536 40010 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38230.8
Current children cumulated vsize (KiB) 6407364

[startup+4842.3 s]
/proc/loadavg: 8.00 8.03 8.00 9/180 4761
/proc/meminfo: memFree=23923424/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) S 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 10666635808867352575 0 0 17 3 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
[pid=4603] ppid=4602 vsize=5004176 CPUtime=4842.39 cores=0
/proc/4603/stat : 4603 (cryptominisat) R 4602 4602 3977 0 -1 4202496 211388 0 0 0 484144 95 0 0 25 0 1 0 606512304 5124276224 33023 33554432000 4194304 4587354 140735640025072 18446744073709551615 4238616 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4603/statm: 1251044 33023 337 96 0 1247606 0
[pid=4604] ppid=4602 vsize=61532 CPUtime=4842.38 cores=1
/proc/4604/stat : 4604 (clasp) R 4602 4602 3977 0 -1 4202496 11973 0 0 0 484235 3 0 0 25 0 1 0 606512304 63008768 11860 33554432000 4194304 4785740 140733420045872 18446744073709551615 4562722 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4604/statm: 15383 11860 341 145 0 12428 0
[pid=4605] ppid=4602 vsize=455964 CPUtime=4842.28 cores=2
/proc/4605/stat : 4605 (TNM) R 4602 4602 3977 0 -1 4202496 365 0 0 0 484227 1 0 0 25 0 1 0 606512304 466907136 262 33554432000 134512640 134534564 4287689392 18446744073709551615 134528652 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4605/statm: 113991 262 103 6 0 112657 0
[pid=4606] ppid=4602 vsize=596028 CPUtime=4842.39 cores=3
/proc/4606/stat : 4606 (march_hi) R 4602 4602 3977 0 -1 4202496 134226 0 0 0 484125 114 0 0 25 0 1 0 606512304 610332672 134119 33554432000 4194304 4331988 140734796245120 18446744073709551615 4237130 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4606/statm: 149007 134119 173 34 0 147449 0
[pid=4607] ppid=4602 vsize=283468 CPUtime=19341.1 cores=4-7
/proc/4607/stat : 4607 (plingeling) S 4602 4602 3977 0 -1 4202496 990632 1250 0 0 1933950 158 0 0 19 0 5 0 606512304 290271232 42986 33554432000 4194304 4356188 140735583514224 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4607/statm: 70867 42986 164 40 0 69411 0
[pid=4607/tid=4614] ppid=4602 vsize=283468 CPUtime=4842.39 cores=4-7
/proc/4607/task/4614/stat : 4614 (plingeling) R 4602 4602 3977 0 -1 4202560 258682 1250 0 0 484194 45 0 0 25 0 5 0 606512305 290271232 42986 33554432000 4194304 4356188 140735583514224 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4607/tid=4615] ppid=4602 vsize=283468 CPUtime=4841.92 cores=4-7
/proc/4607/task/4615/stat : 4615 (plingeling) R 4602 4602 3977 0 -1 4202560 237181 1250 0 0 484157 35 0 0 25 0 5 0 606512305 290271232 42986 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4607/tid=4616] ppid=4602 vsize=283468 CPUtime=4842.37 cores=4-7
/proc/4607/task/4616/stat : 4616 (plingeling) R 4602 4602 3977 0 -1 4202560 245784 1250 0 0 484201 36 0 0 25 0 5 0 606512305 290271232 42986 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4607/tid=4617] ppid=4602 vsize=283468 CPUtime=4814.37 cores=4-7
/proc/4607/task/4617/stat : 4617 (plingeling) R 4602 4602 3977 0 -1 4202560 248452 1250 0 0 481397 40 0 0 25 0 5 0 606512305 290271232 42986 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38710.5
Current children cumulated vsize (KiB) 6419492

[startup+4902.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/180 4763
/proc/meminfo: memFree=23895620/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) S 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 10666635808867352575 0 0 17 3 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
[pid=4603] ppid=4602 vsize=5004244 CPUtime=4902.39 cores=0
/proc/4603/stat : 4603 (cryptominisat) R 4602 4602 3977 0 -1 4202496 216477 0 0 0 490141 98 0 0 25 0 1 0 606512304 5124345856 31513 33554432000 4194304 4587354 140735640025072 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4603/statm: 1251061 31513 337 96 0 1247623 0
[pid=4604] ppid=4602 vsize=61672 CPUtime=4902.38 cores=1
/proc/4604/stat : 4604 (clasp) R 4602 4602 3977 0 -1 4202496 11991 0 0 0 490235 3 0 0 25 0 1 0 606512304 63152128 11878 33554432000 4194304 4785740 140733420045872 18446744073709551615 4287204 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4604/statm: 15418 11878 341 145 0 12463 0
[pid=4605] ppid=4602 vsize=455964 CPUtime=4902.28 cores=2
/proc/4605/stat : 4605 (TNM) R 4602 4602 3977 0 -1 4202496 365 0 0 0 490227 1 0 0 25 0 1 0 606512304 466907136 262 33554432000 134512640 134534564 4287689392 18446744073709551615 134528955 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4605/statm: 113991 262 103 6 0 112657 0
[pid=4606] ppid=4602 vsize=596028 CPUtime=4902.39 cores=3
/proc/4606/stat : 4606 (march_hi) R 4602 4602 3977 0 -1 4202496 135584 0 0 0 490124 115 0 0 25 0 1 0 606512304 610332672 135477 33554432000 4194304 4331988 140734796245120 18446744073709551615 4226471 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4606/statm: 149007 135477 173 34 0 147449 0
[pid=4607] ppid=4602 vsize=314192 CPUtime=19580.8 cores=4-7
/proc/4607/stat : 4607 (plingeling) S 4602 4602 3977 0 -1 4202496 998313 1250 0 0 1957917 159 0 0 19 0 5 0 606512304 321732608 50667 33554432000 4194304 4356188 140735583514224 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4607/statm: 78548 50667 164 40 0 77092 0
[pid=4607/tid=4614] ppid=4602 vsize=314192 CPUtime=4902.39 cores=4-7
/proc/4607/task/4614/stat : 4614 (plingeling) R 4602 4602 3977 0 -1 4202560 261178 1250 0 0 490193 46 0 0 25 0 5 0 606512305 321732608 50667 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4607/tid=4615] ppid=4602 vsize=314192 CPUtime=4901.92 cores=4-7
/proc/4607/task/4615/stat : 4615 (plingeling) R 4602 4602 3977 0 -1 4202560 238877 1250 0 0 490157 35 0 0 25 0 5 0 606512305 321732608 50667 33554432000 4194304 4356188 140735583514224 18446744073709551615 4332023 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4607/tid=4616] ppid=4602 vsize=314192 CPUtime=4902.37 cores=4-7
/proc/4607/task/4616/stat : 4616 (plingeling) R 4602 4602 3977 0 -1 4202560 248217 1250 0 0 490201 36 0 0 25 0 5 0 606512305 321732608 50667 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331019 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4607/tid=4617] ppid=4602 vsize=314192 CPUtime=4874.04 cores=4-7
/proc/4607/task/4617/stat : 4617 (plingeling) R 4602 4602 3977 0 -1 4202560 249508 1250 0 0 487364 40 0 0 25 0 5 0 606512305 321732608 50667 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39190.2
Current children cumulated vsize (KiB) 6450424

[startup+4962.3 s]
/proc/loadavg: 8.05 8.03 8.00 9/180 4764
/proc/meminfo: memFree=23901408/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) S 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 10666635808867352575 0 0 17 3 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
[pid=4603] ppid=4602 vsize=5004244 CPUtime=4962.39 cores=0
/proc/4603/stat : 4603 (cryptominisat) R 4602 4602 3977 0 -1 4202496 220071 0 0 0 496139 100 0 0 25 0 1 0 606512304 5124345856 35000 33554432000 4194304 4587354 140735640025072 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4603/statm: 1251061 35000 337 96 0 1247623 0
[pid=4604] ppid=4602 vsize=61672 CPUtime=4962.38 cores=1
/proc/4604/stat : 4604 (clasp) R 4602 4602 3977 0 -1 4202496 11993 0 0 0 496235 3 0 0 25 0 1 0 606512304 63152128 11880 33554432000 4194304 4785740 140733420045872 18446744073709551615 4286976 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4604/statm: 15418 11880 341 145 0 12463 0
[pid=4605] ppid=4602 vsize=455964 CPUtime=4962.28 cores=2
/proc/4605/stat : 4605 (TNM) R 4602 4602 3977 0 -1 4202496 365 0 0 0 496227 1 0 0 25 0 1 0 606512304 466907136 262 33554432000 134512640 134534564 4287689392 18446744073709551615 134528200 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4605/statm: 113991 262 103 6 0 112657 0
[pid=4606] ppid=4602 vsize=596028 CPUtime=4962.39 cores=3
/proc/4606/stat : 4606 (march_hi) R 4602 4602 3977 0 -1 4202496 136988 0 0 0 496123 116 0 0 25 0 1 0 606512304 610332672 136881 33554432000 4194304 4331988 140734796245120 18446744073709551615 4211509 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4606/statm: 149007 136881 173 34 0 147449 0
[pid=4607] ppid=4602 vsize=286672 CPUtime=19820.4 cores=4-7
/proc/4607/stat : 4607 (plingeling) S 4602 4602 3977 0 -1 4202496 1001609 1250 0 0 1981883 160 0 0 19 0 5 0 606512304 293552128 43787 33554432000 4194304 4356188 140735583514224 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4607/statm: 71668 43787 164 40 0 70212 0
[pid=4607/tid=4614] ppid=4602 vsize=286672 CPUtime=4962.4 cores=4-7
/proc/4607/task/4614/stat : 4614 (plingeling) R 4602 4602 3977 0 -1 4202560 261466 1250 0 0 496194 46 0 0 25 0 5 0 606512305 293552128 43787 33554432000 4194304 4356188 140735583514224 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4607/tid=4615] ppid=4602 vsize=286672 CPUtime=4961.92 cores=4-7
/proc/4607/task/4615/stat : 4615 (plingeling) R 4602 4602 3977 0 -1 4202560 239645 1250 0 0 496156 36 0 0 25 0 5 0 606512305 293552128 43787 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331305 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4607/tid=4616] ppid=4602 vsize=286672 CPUtime=4962.36 cores=4-7
/proc/4607/task/4616/stat : 4616 (plingeling) R 4602 4602 3977 0 -1 4202560 249017 1250 0 0 496200 36 0 0 25 0 5 0 606512305 293552128 43787 33554432000 4194304 4356188 140735583514224 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4607/tid=4617] ppid=4602 vsize=286672 CPUtime=4933.71 cores=4-7
/proc/4607/task/4617/stat : 4617 (plingeling) R 4602 4602 3977 0 -1 4202560 250948 1250 0 0 493331 40 0 0 25 0 5 0 606512305 293552128 43787 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331111 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39669.9
Current children cumulated vsize (KiB) 6422904



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.09 8.04 8.00 9/180 4765
/proc/meminfo: memFree=23890580/32950940 swapFree=67111364/67111528
[pid=4602] ppid=4600 vsize=18324 CPUtime=0 cores=0-7
/proc/4602/stat : 4602 (ppfolio) S 4600 4602 3977 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 606512304 18763776 390 33554432000 4194304 4287625 140733892305072 18446744073709551615 215988612623 0 0 4096 65536 10666635808867352575 0 0 17 3 0 0 0
/proc/4602/statm: 4581 390 341 23 0 91 0
[pid=4603] ppid=4602 vsize=5004328 CPUtime=5000.09 cores=0
/proc/4603/stat : 4603 (cryptominisat) R 4602 4602 3977 0 -1 4202496 224487 0 0 0 499907 102 0 0 25 0 1 0 606512304 5124431872 32764 33554432000 4194304 4587354 140735640025072 18446744073709551615 4238616 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/4603/statm: 1251082 32764 337 96 0 1247644 0
[pid=4604] ppid=4602 vsize=61672 CPUtime=5000.08 cores=1
/proc/4604/stat : 4604 (clasp) R 4602 4602 3977 0 -1 4202496 11994 0 0 0 500005 3 0 0 25 0 1 0 606512304 63152128 11881 33554432000 4194304 4785740 140733420045872 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4604/statm: 15418 11881 341 145 0 12463 0
[pid=4605] ppid=4602 vsize=455964 CPUtime=4999.98 cores=2
/proc/4605/stat : 4605 (TNM) R 4602 4602 3977 0 -1 4202496 365 0 0 0 499997 1 0 0 25 0 1 0 606512304 466907136 262 33554432000 134512640 134534564 4287689392 18446744073709551615 134530468 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/4605/statm: 113991 262 103 6 0 112657 0
[pid=4606] ppid=4602 vsize=596028 CPUtime=5000.1 cores=3
/proc/4606/stat : 4606 (march_hi) R 4602 4602 3977 0 -1 4202496 138010 0 0 0 499893 117 0 0 25 0 1 0 606512304 610332672 137903 33554432000 4194304 4331988 140734796245120 18446744073709551615 4229544 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/4606/statm: 149007 137903 173 34 0 147449 0
[pid=4607] ppid=4602 vsize=303312 CPUtime=19971 cores=4-7
/proc/4607/stat : 4607 (plingeling) S 4602 4602 3977 0 -1 4202496 1013961 1250 0 0 1996943 161 0 0 19 0 5 0 606512304 310591488 47947 33554432000 4194304 4356188 140735583514224 18446744073709551615 216000395237 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/4607/statm: 75828 47947 164 40 0 74372 0
[pid=4607/tid=4614] ppid=4602 vsize=303312 CPUtime=5000.1 cores=4-7
/proc/4607/task/4614/stat : 4614 (plingeling) R 4602 4602 3977 0 -1 4202560 267002 1250 0 0 499964 46 0 0 25 0 5 0 606512305 310591488 47947 33554432000 4194304 4356188 140735583514224 18446744073709551615 4332165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4607/tid=4615] ppid=4602 vsize=303312 CPUtime=4999.61 cores=4-7
/proc/4607/task/4615/stat : 4615 (plingeling) R 4602 4602 3977 0 -1 4202560 241309 1250 0 0 499925 36 0 0 25 0 5 0 606512305 310591488 47947 33554432000 4194304 4356188 140735583514224 18446744073709551615 4330856 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4607/tid=4616] ppid=4602 vsize=303312 CPUtime=5000.07 cores=4-7
/proc/4607/task/4616/stat : 4616 (plingeling) R 4602 4602 3977 0 -1 4202560 249785 1250 0 0 499971 36 0 0 25 0 5 0 606512305 310591488 47947 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4607/tid=4617] ppid=4602 vsize=303312 CPUtime=4971.21 cores=4-7
/proc/4607/task/4617/stat : 4617 (plingeling) R 4602 4602 3977 0 -1 4202560 255332 1250 0 0 497081 40 0 0 25 0 5 0 606512305 310591488 47947 33554432000 4194304 4356188 140735583514224 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39971.3
Current children cumulated vsize (KiB) 6439628

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 4602 and gives
#  childrusage.ru_utime.tv_sec=14999
#  childrusage.ru_utime.tv_usec=121786
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=96833
# CPU time returned by wait4() is 15000.2
# while last known CPU time is 39971.3
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39971.3
CPU user time (s): 39967.4
CPU system time (s): 3.84
CPU usage (%): 799.423
Max. virtual memory (cumulated for all children) (KiB): 11017964

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14999.1
system time used= 1.09683
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 237383
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 907
involuntary context switches= 92151

runsolver used 8.29974 second user time and 18.9761 second system time

The end

Launcher Data 

Begin job on node117 at 2011-04-30 13:26:55
IDJOB=3221847
IDBENCH=82991
IDSOLVER=1638
FILE ID=node117/3221847-1304162815
RUNJOBID= node117-1304157751-3991
PBS_JOBID= 13170912
Free space on /tmp= 72632 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v600-c2556-S506401817-075.UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3221847-1304162815/watcher-3221847-1304162815 -o /tmp/evaluation-result-3221847-1304162815/solver-3221847-1304162815 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3221847-1304162815.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 2d7407b1b64ab354711b83851f166439
RANDOM SEED=1142597143

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      24810956 kB
Buffers:       1893080 kB
Cached:        5618568 kB
SwapCached:        160 kB
Active:         898976 kB
Inactive:      6676292 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      24810956 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:           10108 kB
Writeback:           0 kB
AnonPages:       63728 kB
Mapped:          14772 kB
Slab:           499872 kB
PageTables:       3776 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS:   181568 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72620 MiB
End job on node117 at 2011-04-30 14:50:17