Trace number 3221109

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39972.4 5000.02

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v650-c2769-S960948361-055.UNKNOWN.cnf
MD5SUMb51753a48e6efd66554e7afe4efc3b19
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables650
Number of clauses2769
Sum of the clauses size8307
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32769
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3221109-1304160888/watcher-3221109-1304160888 -o /tmp/evaluation-result-3221109-1304160888/solver-3221109-1304160888 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3221109-1304160888.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.46 7.87 7.94 1/170 12298
/proc/meminfo: memFree=30091152/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0 cores=0-7
/proc/12298/stat : 12298 (ppfolio) R 12296 12298 12272 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0

[startup+0.0665589 s]
/proc/loadavg: 7.46 7.87 7.94 1/170 12298
/proc/meminfo: memFree=30091152/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0 cores=0-7
/proc/12298/stat : 12298 (ppfolio) R 12296 12298 12272 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100552 s]
/proc/loadavg: 7.46 7.87 7.94 1/170 12298
/proc/meminfo: memFree=30091152/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0 cores=0-7
/proc/12298/stat : 12298 (ppfolio) R 12296 12298 12272 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300516 s]
/proc/loadavg: 7.46 7.87 7.94 1/170 12298
/proc/meminfo: memFree=30091152/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0 cores=0-7
/proc/12298/stat : 12298 (ppfolio) S 12296 12298 12272 0 -1 4202496 519 0 0 0 0 0 0 0 23 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 8425962365660430335 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700442 s]
/proc/loadavg: 7.46 7.87 7.94 1/170 12298
/proc/meminfo: memFree=30091152/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0 cores=0-7
/proc/12298/stat : 12298 (ppfolio) S 12296 12298 12272 0 -1 4202496 519 0 0 0 0 0 0 0 19 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 8425962365660430335 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50129 s]
/proc/loadavg: 7.46 7.87 7.94 9/180 12313
/proc/meminfo: memFree=30072200/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0 cores=0-7
/proc/12298/stat : 12298 (ppfolio) S 12296 12298 12272 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 8425962365660430335 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
[pid=12299] ppid=12298 vsize=1236252 CPUtime=1.49 cores=0
/proc/12299/stat : 12299 (cryptominisat) R 12298 12298 12272 0 -1 4202496 4094 0 0 0 149 0 0 0 25 0 1 0 606319471 1265922048 2722 33554432000 4194304 4587354 140737277725648 18446744073709551615 4388277 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12299/statm: 309063 2722 337 96 0 305625 0
[pid=12300] ppid=12298 vsize=14312 CPUtime=1.49 cores=1
/proc/12300/stat : 12300 (clasp) R 12298 12298 12272 0 -1 4202496 1045 0 0 0 149 0 0 0 20 0 1 0 606319471 14655488 932 33554432000 4194304 4785740 140734900104672 18446744073709551615 4553097 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12300/statm: 3578 932 340 145 0 623 0
[pid=12301] ppid=12298 vsize=455960 CPUtime=1.49 cores=2
/proc/12301/stat : 12301 (TNM) R 12298 12298 12272 0 -1 4202496 364 0 0 0 149 0 0 0 20 0 1 0 606319471 466903040 261 33554432000 134512640 134534564 4291496416 18446744073709551615 134528628 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12301/statm: 113990 261 95 6 0 112656 0
[pid=12302] ppid=12298 vsize=8412 CPUtime=1.49 cores=3
/proc/12302/stat : 12302 (march_hi) R 12298 12298 12272 0 -1 4202496 730 0 0 0 149 0 0 0 20 0 1 0 606319471 8613888 604 33554432000 4194304 4331988 140736297594864 18446744073709551615 4229363 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12302/statm: 2103 604 172 34 0 545 0
[pid=12303] ppid=12298 vsize=123788 CPUtime=5.93 cores=4-7
/proc/12303/stat : 12303 (plingeling) S 12298 12298 12272 0 -1 4202496 3868 1255 0 0 593 0 0 0 25 0 5 0 606319471 126758912 3039 33554432000 4194304 4356188 140733487172368 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12303/statm: 30947 3039 164 40 0 29491 0
[pid=12303/tid=12310] ppid=12298 vsize=123788 CPUtime=1.48 cores=4-7
/proc/12303/task/12310/stat : 12310 (plingeling) R 12298 12298 12272 0 -1 4202560 746 1255 0 0 148 0 0 0 25 0 5 0 606319472 126758912 3039 33554432000 4194304 4356188 140733487172368 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12303/tid=12311] ppid=12298 vsize=123788 CPUtime=1.46 cores=4-7
/proc/12303/task/12311/stat : 12311 (plingeling) R 12298 12298 12272 0 -1 4202560 1034 1255 0 0 146 0 0 0 25 0 5 0 606319472 126758912 3039 33554432000 4194304 4356188 140733487172368 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12303/tid=12312] ppid=12298 vsize=123788 CPUtime=1.48 cores=4-7
/proc/12303/task/12312/stat : 12312 (plingeling) R 12298 12298 12272 0 -1 4202560 847 1255 0 0 148 0 0 0 25 0 5 0 606319472 126758912 3039 33554432000 4194304 4356188 140733487172368 18446744073709551615 4281495 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12303/tid=12313] ppid=12298 vsize=123788 CPUtime=1.48 cores=4-7
/proc/12303/task/12313/stat : 12313 (plingeling) R 12298 12298 12272 0 -1 4202560 713 1255 0 0 148 0 0 0 25 0 5 0 606319472 126758912 3039 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.89
Current children cumulated vsize (KiB) 1857048

[startup+3.101 s]
/proc/loadavg: 7.58 7.89 7.94 9/180 12313
/proc/meminfo: memFree=30063644/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0 cores=0-7
/proc/12298/stat : 12298 (ppfolio) S 12296 12298 12272 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 8425962365660430335 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
[pid=12299] ppid=12298 vsize=1240776 CPUtime=3.08 cores=0
/proc/12299/stat : 12299 (cryptominisat) R 12298 12298 12272 0 -1 4202496 5923 0 0 0 308 0 0 0 25 0 1 0 606319471 1270554624 3569 33554432000 4194304 4587354 140737277725648 18446744073709551615 4357671 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12299/statm: 310194 3569 337 96 0 306756 0
[pid=12300] ppid=12298 vsize=15244 CPUtime=3.09 cores=1
/proc/12300/stat : 12300 (clasp) R 12298 12298 12272 0 -1 4202496 1263 0 0 0 309 0 0 0 25 0 1 0 606319471 15609856 1150 33554432000 4194304 4785740 140734900104672 18446744073709551615 4278129 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12300/statm: 3811 1150 340 145 0 856 0
[pid=12301] ppid=12298 vsize=455960 CPUtime=3.09 cores=2
/proc/12301/stat : 12301 (TNM) R 12298 12298 12272 0 -1 4202496 364 0 0 0 309 0 0 0 25 0 1 0 606319471 466903040 261 33554432000 134512640 134534564 4291496416 18446744073709551615 8800545 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12301/statm: 113990 261 95 6 0 112656 0
[pid=12302] ppid=12298 vsize=8792 CPUtime=3.09 cores=3
/proc/12302/stat : 12302 (march_hi) R 12298 12298 12272 0 -1 4202496 789 0 0 0 309 0 0 0 25 0 1 0 606319471 9003008 663 33554432000 4194304 4331988 140736297594864 18446744073709551615 4246727 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12302/statm: 2198 663 173 34 0 640 0
[pid=12303] ppid=12298 vsize=123664 CPUtime=12.32 cores=4-7
/proc/12303/stat : 12303 (plingeling) S 12298 12298 12272 0 -1 4202496 4093 1255 0 0 1232 0 0 0 25 0 5 0 606319471 126631936 3008 33554432000 4194304 4356188 140733487172368 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12303/statm: 30916 3008 164 40 0 29460 0
[pid=12303/tid=12310] ppid=12298 vsize=123664 CPUtime=3.08 cores=4-7
/proc/12303/task/12310/stat : 12310 (plingeling) R 12298 12298 12272 0 -1 4202560 778 1255 0 0 308 0 0 0 25 0 5 0 606319472 126631936 3008 33554432000 4194304 4356188 140733487172368 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12303/tid=12311] ppid=12298 vsize=123664 CPUtime=3.06 cores=4-7
/proc/12303/task/12311/stat : 12311 (plingeling) R 12298 12298 12272 0 -1 4202560 1066 1255 0 0 306 0 0 0 25 0 5 0 606319472 126631936 3008 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12303/tid=12312] ppid=12298 vsize=123664 CPUtime=3.08 cores=4-7
/proc/12303/task/12312/stat : 12312 (plingeling) R 12298 12298 12272 0 -1 4202560 943 1255 0 0 308 0 0 0 25 0 5 0 606319472 126631936 3008 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331811 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12303/tid=12313] ppid=12298 vsize=123664 CPUtime=3.08 cores=4-7
/proc/12303/task/12313/stat : 12313 (plingeling) R 12298 12298 12272 0 -1 4202560 778 1255 0 0 308 0 0 0 25 0 5 0 606319472 126631936 3008 33554432000 4194304 4356188 140733487172368 18446744073709551615 4210791 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.67
Current children cumulated vsize (KiB) 1862760

[startup+6.30042 s]
/proc/loadavg: 7.58 7.89 7.94 9/180 12313
/proc/meminfo: memFree=30053728/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0 cores=0-7
/proc/12298/stat : 12298 (ppfolio) S 12296 12298 12272 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 8425962365660430335 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
[pid=12299] ppid=12298 vsize=1240640 CPUtime=6.28 cores=0
/proc/12299/stat : 12299 (cryptominisat) R 12298 12298 12272 0 -1 4202496 8275 0 0 0 628 0 0 0 25 0 1 0 606319471 1270415360 4058 33554432000 4194304 4587354 140737277725648 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12299/statm: 310160 4058 337 96 0 306722 0
[pid=12300] ppid=12298 vsize=16452 CPUtime=6.29 cores=1
/proc/12300/stat : 12300 (clasp) R 12298 12298 12272 0 -1 4202496 1574 0 0 0 629 0 0 0 25 0 1 0 606319471 16846848 1461 33554432000 4194304 4785740 140734900104672 18446744073709551615 4553032 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12300/statm: 4113 1461 341 145 0 1158 0
[pid=12301] ppid=12298 vsize=455960 CPUtime=6.29 cores=2
/proc/12301/stat : 12301 (TNM) R 12298 12298 12272 0 -1 4202496 364 0 0 0 629 0 0 0 25 0 1 0 606319471 466903040 261 33554432000 134512640 134534564 4291496416 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12301/statm: 113990 261 95 6 0 112656 0
[pid=12302] ppid=12298 vsize=9280 CPUtime=6.29 cores=3
/proc/12302/stat : 12302 (march_hi) R 12298 12298 12272 0 -1 4202496 936 0 0 0 629 0 0 0 25 0 1 0 606319471 9502720 810 33554432000 4194304 4331988 140736297594864 18446744073709551615 4240519 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12302/statm: 2320 810 173 34 0 762 0
[pid=12303] ppid=12298 vsize=124708 CPUtime=25.1 cores=4-7
/proc/12303/stat : 12303 (plingeling) S 12298 12298 12272 0 -1 4202496 6466 1255 0 0 2510 0 0 0 25 0 5 0 606319471 127700992 3269 33554432000 4194304 4356188 140733487172368 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12303/statm: 31177 3269 164 40 0 29721 0
[pid=12303/tid=12310] ppid=12298 vsize=124708 CPUtime=6.28 cores=4-7
/proc/12303/task/12310/stat : 12310 (plingeling) R 12298 12298 12272 0 -1 4202560 1258 1255 0 0 628 0 0 0 25 0 5 0 606319472 127700992 3269 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331319 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12303/tid=12311] ppid=12298 vsize=124708 CPUtime=6.24 cores=4-7
/proc/12303/task/12311/stat : 12311 (plingeling) R 12298 12298 12272 0 -1 4202560 1643 1255 0 0 624 0 0 0 25 0 5 0 606319472 127700992 3269 33554432000 4194304 4356188 140733487172368 18446744073709551615 4332322 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12303/tid=12312] ppid=12298 vsize=124708 CPUtime=6.28 cores=4-7
/proc/12303/task/12312/stat : 12312 (plingeling) R 12298 12298 12272 0 -1 4202560 1681 1255 0 0 628 0 0 0 25 0 5 0 606319472 127700992 3269 33554432000 4194304 4356188 140733487172368 18446744073709551615 4281480 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12303/tid=12313] ppid=12298 vsize=124708 CPUtime=6.27 cores=4-7
/proc/12303/task/12313/stat : 12313 (plingeling) R 12298 12298 12272 0 -1 4202560 1356 1255 0 0 627 0 0 0 25 0 5 0 606319472 127700992 3269 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331335 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.25
Current children cumulated vsize (KiB) 1865364

[startup+12.7012 s]
/proc/loadavg: 7.65 7.89 7.94 9/180 12314
/proc/meminfo: memFree=30043220/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0 cores=0-7
/proc/12298/stat : 12298 (ppfolio) S 12296 12298 12272 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 8425962365660430335 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
[pid=12299] ppid=12298 vsize=1244588 CPUtime=12.69 cores=0
/proc/12299/stat : 12299 (cryptominisat) R 12298 12298 12272 0 -1 4202496 12863 0 0 0 1268 1 0 0 25 0 1 0 606319471 1274458112 4622 33554432000 4194304 4587354 140737277725648 18446744073709551615 4238392 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12299/statm: 311147 4622 337 96 0 307709 0
[pid=12300] ppid=12298 vsize=17956 CPUtime=12.69 cores=1
/proc/12300/stat : 12300 (clasp) R 12298 12298 12272 0 -1 4202496 1928 0 0 0 1269 0 0 0 25 0 1 0 606319471 18386944 1815 33554432000 4194304 4785740 140734900104672 18446744073709551615 4552503 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12300/statm: 4489 1815 341 145 0 1534 0
[pid=12301] ppid=12298 vsize=455960 CPUtime=12.69 cores=2
/proc/12301/stat : 12301 (TNM) R 12298 12298 12272 0 -1 4202496 364 0 0 0 1269 0 0 0 25 0 1 0 606319471 466903040 261 33554432000 134512640 134534564 4291496416 18446744073709551615 134530227 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12301/statm: 113990 261 95 6 0 112656 0
[pid=12302] ppid=12298 vsize=10704 CPUtime=12.69 cores=3
/proc/12302/stat : 12302 (march_hi) R 12298 12298 12272 0 -1 4202496 1184 0 0 0 1269 0 0 0 25 0 1 0 606319471 10960896 1058 33554432000 4194304 4331988 140736297594864 18446744073709551615 4211529 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12302/statm: 2676 1058 173 34 0 1118 0
[pid=12303] ppid=12298 vsize=130492 CPUtime=50.67 cores=4-7
/proc/12303/stat : 12303 (plingeling) S 12298 12298 12272 0 -1 4202496 11624 1255 0 0 5066 1 0 0 25 0 5 0 606319471 133623808 4715 33554432000 4194304 4356188 140733487172368 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12303/statm: 32623 4715 164 40 0 31167 0
[pid=12303/tid=12310] ppid=12298 vsize=130492 CPUtime=12.68 cores=4-7
/proc/12303/task/12310/stat : 12310 (plingeling) R 12298 12298 12272 0 -1 4202560 2476 1255 0 0 1268 0 0 0 25 0 5 0 606319472 133623808 4715 33554432000 4194304 4356188 140733487172368 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12303/tid=12311] ppid=12298 vsize=130492 CPUtime=12.61 cores=4-7
/proc/12303/task/12311/stat : 12311 (plingeling) R 12298 12298 12272 0 -1 4202560 2956 1255 0 0 1261 0 0 0 25 0 5 0 606319472 133623808 4715 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12303/tid=12312] ppid=12298 vsize=130492 CPUtime=12.68 cores=4-7
/proc/12303/task/12312/stat : 12312 (plingeling) R 12298 12298 12272 0 -1 4202560 3058 1255 0 0 1268 0 0 0 25 0 5 0 606319472 133623808 4715 33554432000 4194304 4356188 140733487172368 18446744073709551615 4210830 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12303/tid=12313] ppid=12298 vsize=130492 CPUtime=12.67 cores=4-7
/proc/12303/task/12313/stat : 12313 (plingeling) R 12298 12298 12272 0 -1 4202560 2606 1255 0 0 1267 0 0 0 25 0 5 0 606319472 133623808 4715 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331383 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 101.43
Current children cumulated vsize (KiB) 1878024

[startup+25.5009 s]
/proc/loadavg: 7.70 7.90 7.94 9/180 12314
/proc/meminfo: memFree=30025740/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0 cores=0-7
/proc/12298/stat : 12298 (ppfolio) S 12296 12298 12272 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 8425962365660430335 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
[pid=12299] ppid=12298 vsize=1249852 CPUtime=25.49 cores=0
/proc/12299/stat : 12299 (cryptominisat) R 12298 12298 12272 0 -1 4202496 19644 0 0 0 2546 3 0 0 25 0 1 0 606319471 1279848448 6099 33554432000 4194304 4587354 140737277725648 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=12301] ppid=12298 vsize=455964 CPUtime=4781.62 cores=2
/proc/12301/stat : 12301 (TNM) R 12298 12298 12272 0 -1 4202496 373 0 0 0 478161 1 0 0 25 0 1 0 606319471 466907136 270 33554432000 134512640 134534564 4291496416 18446744073709551615 8800708 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12301/statm: 113991 270 103 6 0 112657 0
[pid=12302] ppid=12298 vsize=830776 CPUtime=4782.43 cores=3
/proc/12302/stat : 12302 (march_hi) R 12298 12298 12272 0 -1 4202496 150157 0 0 0 478128 115 0 0 25 0 1 0 606319471 850714624 150031 33554432000 4194304 4331988 140736297594864 18446744073709551615 4240270 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12302/statm: 207694 150031 173 34 0 206136 0
[pid=12303] ppid=12298 vsize=276956 CPUtime=19103 cores=4-7
/proc/12303/stat : 12303 (plingeling) S 12298 12298 12272 0 -1 4202496 882811 1255 0 0 1910171 132 0 0 25 0 5 0 606319471 283602944 41331 33554432000 4194304 4356188 140733487172368 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12303/statm: 69239 41331 164 40 0 67783 0
[pid=12303/tid=12310] ppid=12298 vsize=276956 CPUtime=4782.43 cores=4-7
/proc/12303/task/12310/stat : 12310 (plingeling) R 12298 12298 12272 0 -1 4202560 222416 1255 0 0 478210 33 0 0 25 0 5 0 606319472 283602944 41331 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331072 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12303/tid=12311] ppid=12298 vsize=276956 CPUtime=4756.33 cores=4-7
/proc/12303/task/12311/stat : 12311 (plingeling) R 12298 12298 12272 0 -1 4202560 222705 1255 0 0 475602 31 0 0 25 0 5 0 606319472 283602944 41331 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12303/tid=12312] ppid=12298 vsize=276956 CPUtime=4781.96 cores=4-7
/proc/12303/task/12312/stat : 12312 (plingeling) R 12298 12298 12272 0 -1 4202560 222040 1255 0 0 478163 33 0 0 25 0 5 0 606319472 283602944 41331 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12303/tid=12313] ppid=12298 vsize=276956 CPUtime=4782.27 cores=4-7
/proc/12303/task/12313/stat : 12313 (plingeling) R 12298 12298 12272 0 -1 4202560 215122 1255 0 0 478194 33 0 0 25 0 5 0 606319472 283602944 41331 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38231.9
Current children cumulated vsize (KiB) 3017048

[startup+4842.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/180 12463
/proc/meminfo: memFree=29116872/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0 cores=0-7
/proc/12298/stat : 12298 (ppfolio) S 12296 12298 12272 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 8425962365660430335 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
[pid=12299] ppid=12298 vsize=1369564 CPUtime=4842.42 cores=0
/proc/12299/stat : 12299 (cryptominisat) R 12298 12298 12272 0 -1 4202496 360828 0 0 0 484077 165 0 0 25 0 1 0 606319471 1402433536 39506 33554432000 4194304 4587354 140737277725648 18446744073709551615 4238573 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12299/statm: 342391 39506 337 96 0 338953 0
[pid=12300] ppid=12298 vsize=65464 CPUtime=4842.4 cores=1
/proc/12300/stat : 12300 (clasp) R 12298 12298 12272 0 -1 4202496 12839 0 0 0 484237 3 0 0 25 0 1 0 606319471 67035136 12726 33554432000 4194304 4785740 140734900104672 18446744073709551615 4287053 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12300/statm: 16366 12726 341 145 0 13411 0
[pid=12301] ppid=12298 vsize=455964 CPUtime=4841.63 cores=2
/proc/12301/stat : 12301 (TNM) R 12298 12298 12272 0 -1 4202496 373 0 0 0 484162 1 0 0 25 0 1 0 606319471 466907136 270 33554432000 134512640 134534564 4291496416 18446744073709551615 134528273 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12301/statm: 113991 270 103 6 0 112657 0
[pid=12302] ppid=12298 vsize=830776 CPUtime=4842.44 cores=3
/proc/12302/stat : 12302 (march_hi) R 12298 12298 12272 0 -1 4202496 151979 0 0 0 484126 118 0 0 25 0 1 0 606319471 850714624 151853 33554432000 4194304 4331988 140736297594864 18446744073709551615 4226420 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12302/statm: 207694 151853 173 34 0 206136 0
[pid=12303] ppid=12298 vsize=269532 CPUtime=19342.7 cores=4-7
/proc/12303/stat : 12303 (plingeling) S 12298 12298 12272 0 -1 4202496 888251 1255 0 0 1934138 133 0 0 25 0 5 0 606319471 276000768 39475 33554432000 4194304 4356188 140733487172368 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12303/statm: 67383 39475 164 40 0 65927 0
[pid=12303/tid=12310] ppid=12298 vsize=269532 CPUtime=4842.43 cores=4-7
/proc/12303/task/12310/stat : 12310 (plingeling) R 12298 12298 12272 0 -1 4202560 223568 1255 0 0 484210 33 0 0 25 0 5 0 606319472 276000768 39475 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331268 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12303/tid=12311] ppid=12298 vsize=269532 CPUtime=4816 cores=4-7
/proc/12303/task/12311/stat : 12311 (plingeling) R 12298 12298 12272 0 -1 4202560 224145 1255 0 0 481569 31 0 0 25 0 5 0 606319472 276000768 39475 33554432000 4194304 4356188 140733487172368 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12303/tid=12312] ppid=12298 vsize=269532 CPUtime=4841.96 cores=4-7
/proc/12303/task/12312/stat : 12312 (plingeling) R 12298 12298 12272 0 -1 4202560 224248 1255 0 0 484162 34 0 0 25 0 5 0 606319472 276000768 39475 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12303/tid=12313] ppid=12298 vsize=269532 CPUtime=4842.27 cores=4-7
/proc/12303/task/12313/stat : 12313 (plingeling) R 12298 12298 12272 0 -1 4202560 215762 1255 0 0 484194 33 0 0 25 0 5 0 606319472 276000768 39475 33554432000 4194304 4356188 140733487172368 18446744073709551615 4332165 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38711.6
Current children cumulated vsize (KiB) 3009624

[startup+4902.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/180 12465
/proc/meminfo: memFree=29088704/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12298/stat : 12298 (ppfolio) S 12296 12298 12272 0 -1 4202496 519 0 0 0 0 1 0 0 15 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
[pid=12299] ppid=12298 vsize=1373732 CPUtime=4902.42 cores=0
/proc/12299/stat : 12299 (cryptominisat) R 12298 12298 12272 0 -1 4202496 361377 0 0 0 490077 165 0 0 25 0 1 0 606319471 1406701568 37746 33554432000 4194304 4587354 140737277725648 18446744073709551615 4238668 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12299/statm: 343433 37746 337 96 0 339995 0
[pid=12300] ppid=12298 vsize=65604 CPUtime=4902.4 cores=1
/proc/12300/stat : 12300 (clasp) R 12298 12298 12272 0 -1 4202496 12868 0 0 0 490237 3 0 0 25 0 1 0 606319471 67178496 12755 33554432000 4194304 4785740 140734900104672 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12300/statm: 16401 12755 341 145 0 13446 0
[pid=12301] ppid=12298 vsize=455964 CPUtime=4901.63 cores=2
/proc/12301/stat : 12301 (TNM) R 12298 12298 12272 0 -1 4202496 373 0 0 0 490162 1 0 0 25 0 1 0 606319471 466907136 270 33554432000 134512640 134534564 4291496416 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12301/statm: 113991 270 103 6 0 112657 0
[pid=12302] ppid=12298 vsize=830776 CPUtime=4902.44 cores=3
/proc/12302/stat : 12302 (march_hi) R 12298 12298 12272 0 -1 4202496 153732 0 0 0 490124 120 0 0 25 0 1 0 606319471 850714624 153606 33554432000 4194304 4331988 140736297594864 18446744073709551615 4211456 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12302/statm: 207694 153606 173 34 0 206136 0
[pid=12303] ppid=12298 vsize=297948 CPUtime=19582.4 cores=4-7
/proc/12303/stat : 12303 (plingeling) S 12298 12298 12272 0 -1 4202496 897787 1255 0 0 1958104 133 0 0 25 0 5 0 606319471 305098752 46579 33554432000 4194304 4356188 140733487172368 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12303/statm: 74487 46579 164 40 0 73031 0
[pid=12303/tid=12310] ppid=12298 vsize=297948 CPUtime=4902.43 cores=4-7
/proc/12303/task/12310/stat : 12310 (plingeling) R 12298 12298 12272 0 -1 4202560 224848 1255 0 0 490210 33 0 0 25 0 5 0 606319472 305098752 46579 33554432000 4194304 4356188 140733487172368 18446744073709551615 4332165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12303/tid=12311] ppid=12298 vsize=297948 CPUtime=4875.69 cores=4-7
/proc/12303/task/12311/stat : 12311 (plingeling) R 12298 12298 12272 0 -1 4202560 229905 1255 0 0 487537 32 0 0 25 0 5 0 606319472 305098752 46579 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12303/tid=12312] ppid=12298 vsize=297948 CPUtime=4901.95 cores=4-7
/proc/12303/task/12312/stat : 12312 (plingeling) R 12298 12298 12272 0 -1 4202560 224536 1255 0 0 490161 34 0 0 25 0 5 0 606319472 305098752 46579 33554432000 4194304 4356188 140733487172368 18446744073709551615 4332026 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12303/tid=12313] ppid=12298 vsize=297948 CPUtime=4902.27 cores=4-7
/proc/12303/task/12313/stat : 12313 (plingeling) R 12298 12298 12272 0 -1 4202560 217970 1255 0 0 490194 33 0 0 25 0 5 0 606319472 305098752 46579 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39191.3
Current children cumulated vsize (KiB) 3042348

[startup+4962.3 s]
/proc/loadavg: 8.00 8.01 8.00 9/180 12467
/proc/meminfo: memFree=29083608/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12298/stat : 12298 (ppfolio) S 12296 12298 12272 0 -1 4202496 519 0 0 0 0 1 0 0 15 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
[pid=12299] ppid=12298 vsize=1388344 CPUtime=4962.43 cores=0
/proc/12299/stat : 12299 (cryptominisat) R 12298 12298 12272 0 -1 4202496 366005 0 0 0 496076 167 0 0 25 0 1 0 606319471 1421664256 40086 33554432000 4194304 4587354 140737277725648 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12299/statm: 347086 40086 337 96 0 343648 0
[pid=12300] ppid=12298 vsize=65772 CPUtime=4962.41 cores=1
/proc/12300/stat : 12300 (clasp) R 12298 12298 12272 0 -1 4202496 12886 0 0 0 496238 3 0 0 25 0 1 0 606319471 67350528 12773 33554432000 4194304 4785740 140734900104672 18446744073709551615 4553411 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12300/statm: 16443 12773 341 145 0 13488 0
[pid=12301] ppid=12298 vsize=455964 CPUtime=4961.63 cores=2
/proc/12301/stat : 12301 (TNM) R 12298 12298 12272 0 -1 4202496 373 0 0 0 496162 1 0 0 25 0 1 0 606319471 466907136 270 33554432000 134512640 134534564 4291496416 18446744073709551615 8800655 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12301/statm: 113991 270 103 6 0 112657 0
[pid=12302] ppid=12298 vsize=830776 CPUtime=4962.44 cores=3
/proc/12302/stat : 12302 (march_hi) R 12298 12298 12272 0 -1 4202496 155486 0 0 0 496123 121 0 0 25 0 1 0 606319471 850714624 155360 33554432000 4194304 4331988 140736297594864 18446744073709551615 4229456 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12302/statm: 207694 155360 173 34 0 206136 0
[pid=12303] ppid=12298 vsize=285916 CPUtime=19822.1 cores=4-7
/proc/12303/stat : 12303 (plingeling) S 12298 12298 12272 0 -1 4202496 906299 1255 0 0 1982070 135 0 0 25 0 5 0 606319471 292777984 43571 33554432000 4194304 4356188 140733487172368 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12303/statm: 71479 43571 164 40 0 70023 0
[pid=12303/tid=12310] ppid=12298 vsize=285916 CPUtime=4962.43 cores=4-7
/proc/12303/task/12310/stat : 12310 (plingeling) R 12298 12298 12272 0 -1 4202560 226000 1255 0 0 496210 33 0 0 25 0 5 0 606319472 292777984 43571 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12303/tid=12311] ppid=12298 vsize=285916 CPUtime=4935.36 cores=4-7
/proc/12303/task/12311/stat : 12311 (plingeling) R 12298 12298 12272 0 -1 4202560 230705 1255 0 0 493504 32 0 0 25 0 5 0 606319472 292777984 43571 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12303/tid=12312] ppid=12298 vsize=285916 CPUtime=4961.94 cores=4-7
/proc/12303/task/12312/stat : 12312 (plingeling) R 12298 12298 12272 0 -1 4202560 230840 1255 0 0 496160 34 0 0 25 0 5 0 606319472 292777984 43571 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12303/tid=12313] ppid=12298 vsize=285916 CPUtime=4962.27 cores=4-7
/proc/12303/task/12313/stat : 12313 (plingeling) R 12298 12298 12272 0 -1 4202560 218226 1255 0 0 496194 33 0 0 25 0 5 0 606319472 292777984 43571 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39671
Current children cumulated vsize (KiB) 3045096



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.00 8.01 8.00 9/180 12468
/proc/meminfo: memFree=29056816/32950928 swapFree=67054472/67111528
[pid=12298] ppid=12296 vsize=18324 CPUtime=0.01 cores=0-7
/proc/12298/stat : 12298 (ppfolio) S 12296 12298 12272 0 -1 4202496 519 0 0 0 0 1 0 0 15 0 1 0 606319471 18763776 389 33554432000 4194304 4287625 140733483105520 18446744073709551615 260905414159 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/12298/statm: 4581 389 341 23 0 91 0
[pid=12299] ppid=12298 vsize=1388344 CPUtime=5000.13 cores=0
/proc/12299/stat : 12299 (cryptominisat) R 12298 12298 12272 0 -1 4202496 368251 0 0 0 499845 168 0 0 25 0 1 0 606319471 1421664256 42260 33554432000 4194304 4587354 140737277725648 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12299/statm: 347086 42260 337 96 0 343648 0
[pid=12300] ppid=12298 vsize=65772 CPUtime=5000.11 cores=1
/proc/12300/stat : 12300 (clasp) R 12298 12298 12272 0 -1 4202496 12888 0 0 0 500008 3 0 0 25 0 1 0 606319471 67350528 12775 33554432000 4194304 4785740 140734900104672 18446744073709551615 4553344 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12300/statm: 16443 12775 341 145 0 13488 0
[pid=12301] ppid=12298 vsize=455964 CPUtime=4999.34 cores=2
/proc/12301/stat : 12301 (TNM) R 12298 12298 12272 0 -1 4202496 373 0 0 0 499933 1 0 0 25 0 1 0 606319471 466907136 270 33554432000 134512640 134534564 4291496416 18446744073709551615 134528633 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12301/statm: 113991 270 103 6 0 112657 0
[pid=12302] ppid=12298 vsize=830776 CPUtime=5000.15 cores=3
/proc/12302/stat : 12302 (march_hi) R 12298 12298 12272 0 -1 4202496 156622 0 0 0 499893 122 0 0 25 0 1 0 606319471 850714624 156496 33554432000 4194304 4331988 140736297594864 18446744073709551615 4209134 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12302/statm: 207694 156496 173 34 0 206136 0
[pid=12303] ppid=12298 vsize=300252 CPUtime=19972.7 cores=4-7
/proc/12303/stat : 12303 (plingeling) S 12298 12298 12272 0 -1 4202496 909883 1255 0 0 1997131 135 0 0 25 0 5 0 606319471 307458048 47155 33554432000 4194304 4356188 140733487172368 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12303/statm: 75063 47155 164 40 0 73607 0
[pid=12303/tid=12310] ppid=12298 vsize=300252 CPUtime=5000.14 cores=4-7
/proc/12303/task/12310/stat : 12310 (plingeling) R 12298 12298 12272 0 -1 4202560 227280 1255 0 0 499981 33 0 0 25 0 5 0 606319472 307458048 47155 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12303/tid=12311] ppid=12298 vsize=300252 CPUtime=4972.85 cores=4-7
/proc/12303/task/12311/stat : 12311 (plingeling) R 12298 12298 12272 0 -1 4202560 231857 1255 0 0 497253 32 0 0 25 0 5 0 606319472 307458048 47155 33554432000 4194304 4356188 140733487172368 18446744073709551615 4332109 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12303/tid=12312] ppid=12298 vsize=300252 CPUtime=4999.64 cores=4-7
/proc/12303/task/12312/stat : 12312 (plingeling) R 12298 12298 12272 0 -1 4202560 230840 1255 0 0 499930 34 0 0 25 0 5 0 606319472 307458048 47155 33554432000 4194304 4356188 140733487172368 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=12303/tid=12313] ppid=12298 vsize=300252 CPUtime=4999.98 cores=4-7
/proc/12303/task/12313/stat : 12313 (plingeling) R 12298 12298 12272 0 -1 4202560 219378 1255 0 0 499965 33 0 0 25 0 5 0 606319472 307458048 47155 33554432000 4194304 4356188 140733487172368 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39972.4
Current children cumulated vsize (KiB) 3059432

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 12298 and gives
#  childrusage.ru_utime.tv_sec=14997
#  childrusage.ru_utime.tv_usec=875976
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=771730
# CPU time returned by wait4() is 14999.6
# while last known CPU time is 39972.4
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39972.4
CPU user time (s): 39968.1
CPU system time (s): 4.3
CPU usage (%): 799.445
Max. virtual memory (cumulated for all children) (KiB): 4222528

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14997.9
system time used= 1.77173
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 382046
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 736
involuntary context switches= 39555

runsolver used 7.64984 second user time and 19.633 second system time

The end

Launcher Data 

Begin job on node116 at 2011-04-30 12:54:48
IDJOB=3221109
IDBENCH=82868
IDSOLVER=1638
FILE ID=node116/3221109-1304160888
RUNJOBID= node116-1304160888-12286
PBS_JOBID= 13170926
Free space on /tmp= 73288 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v650-c2769-S960948361-055.UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3221109-1304160888/watcher-3221109-1304160888 -o /tmp/evaluation-result-3221109-1304160888/solver-3221109-1304160888 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3221109-1304160888.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= b51753a48e6efd66554e7afe4efc3b19
RANDOM SEED=1821729124

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30091680 kB
Buffers:       1014688 kB
Cached:        1487088 kB
SwapCached:       9656 kB
Active:         170784 kB
Inactive:      2348576 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30091680 kB
SwapTotal:    67111528 kB
SwapFree:     67054472 kB
Dirty:           10196 kB
Writeback:           0 kB
AnonPages:       15668 kB
Mapped:          12420 kB
Slab:           275768 kB
PageTables:       4300 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   184500 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73276 MiB
End job on node116 at 2011-04-30 14:18:10