Trace number 3275801

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 40000.5 5003.62

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/
VanderWaerden3k/VanDerWaerden_2-3-19_349.cnf
MD5SUMfaed4f95ee485d18812f5d39a28df3e5
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables349
Number of clauses33487
Sum of the clauses size151837
Maximum clause length19
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 330276
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 53211

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3275801-1304170825/watcher-3275801-1304170825 -o /tmp/evaluation-result-3275801-1304170825/solver-3275801-1304170825 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3275801-1304170825.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.37 7.88 7.96 2/170 23777
/proc/meminfo: memFree=30798476/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 25 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 6 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0

[startup+0.0448351 s]
/proc/loadavg: 7.37 7.88 7.96 2/170 23777
/proc/meminfo: memFree=30798476/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 25 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 6 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.157815 s]
/proc/loadavg: 7.37 7.88 7.96 2/170 23777
/proc/meminfo: memFree=30798476/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 25 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 6 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.357778 s]
/proc/loadavg: 7.37 7.88 7.96 2/170 23777
/proc/meminfo: memFree=30798476/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 24 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 6 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700724 s]
/proc/loadavg: 7.37 7.88 7.96 2/170 23777
/proc/meminfo: memFree=30798476/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 24 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 6 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50059 s]
/proc/loadavg: 7.37 7.88 7.96 9/180 23792
/proc/meminfo: memFree=30762140/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 6 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
[pid=23778] ppid=23777 vsize=1636568 CPUtime=1.48 cores=0
/proc/23778/stat : 23778 (cryptominisat) R 23777 23777 23522 0 -1 4202496 5104 0 0 0 148 0 0 0 25 0 1 0 528681190 1675845632 2013 33554432000 4194304 4587354 140733422818288 18446744073709551615 4239244 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23778/statm: 409142 2013 327 96 0 405704 0
[pid=23779] ppid=23777 vsize=18152 CPUtime=1.48 cores=1
/proc/23779/stat : 23779 (clasp) R 23777 23777 23522 0 -1 4202496 2145 0 0 0 148 0 0 0 25 0 1 0 528681190 18587648 1878 33554432000 4194304 4785740 140734764014592 18446744073709551615 4553032 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/23779/statm: 4538 1878 340 145 0 1583 0
[pid=23780] ppid=23777 vsize=454768 CPUtime=1.49 cores=2
/proc/23780/stat : 23780 (TNM) R 23777 23777 23522 0 -1 4202496 1117 0 0 0 149 0 0 0 20 0 1 0 528681190 465682432 1014 33554432000 134512640 134534564 4286602496 18446744073709551615 134524664 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/23780/statm: 113692 1014 91 6 0 112358 0
[pid=23781] ppid=23777 vsize=17540 CPUtime=1.48 cores=3
/proc/23781/stat : 23781 (march_hi) R 23777 23777 23522 0 -1 4202496 5725 0 0 0 148 0 0 0 20 0 1 0 528681190 17960960 2564 33554432000 4194304 4331988 140733372199024 18446744073709551615 251813900997 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/23781/statm: 4385 2564 151 34 0 2827 0
[pid=23782] ppid=23777 vsize=127376 CPUtime=5.82 cores=4-7
/proc/23782/stat : 23782 (plingeling) S 23777 23777 23522 0 -1 4202496 5561 1259 0 0 582 0 0 0 19 0 5 0 528681190 130433024 3777 33554432000 4194304 4356188 140737205632000 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23782/statm: 31844 3790 162 40 0 30388 0
[pid=23782/tid=23789] ppid=23777 vsize=127376 CPUtime=1.45 cores=4-7
/proc/23782/task/23789/stat : 23789 (plingeling) R 23777 23777 23522 0 -1 4202560 520 1259 0 0 145 0 0 0 23 0 5 0 528681195 130433024 3801 33554432000 4194304 4356188 140737205632000 18446744073709551615 4233996 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23782/tid=23790] ppid=23777 vsize=127376 CPUtime=1.45 cores=4-7
/proc/23782/task/23790/stat : 23790 (plingeling) R 23777 23777 23522 0 -1 4202560 822 1259 0 0 145 0 0 0 23 0 5 0 528681195 130433024 3816 33554432000 4194304 4356188 140737205632000 18446744073709551615 251813934163 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23782/tid=23791] ppid=23777 vsize=127376 CPUtime=1.44 cores=4-7
/proc/23782/task/23791/stat : 23791 (plingeling) R 23777 23777 23522 0 -1 4202560 549 1259 0 0 144 0 0 0 25 0 5 0 528681195 130433024 3832 33554432000 4194304 4356188 140737205632000 18446744073709551615 4334628 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23782/tid=23792] ppid=23777 vsize=127376 CPUtime=1.43 cores=4-7
/proc/23782/task/23792/stat : 23792 (plingeling) R 23777 23777 23522 0 -1 4202560 938 1259 0 0 143 0 0 0 25 0 5 0 528681195 130433024 3845 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331076 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.75
Current children cumulated vsize (KiB) 2272728

[startup+3.10132 s]
/proc/loadavg: 7.37 7.88 7.96 9/180 23792
/proc/meminfo: memFree=30744904/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 6 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
[pid=23778] ppid=23777 vsize=1643460 CPUtime=3.09 cores=0
/proc/23778/stat : 23778 (cryptominisat) R 23777 23777 23522 0 -1 4202496 8069 0 0 0 308 1 0 0 25 0 1 0 528681190 1682903040 3454 33554432000 4194304 4587354 140733422818288 18446744073709551615 4356717 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23778/statm: 410865 3454 337 96 0 407427 0
[pid=23779] ppid=23777 vsize=19652 CPUtime=3.09 cores=1
/proc/23779/stat : 23779 (clasp) R 23777 23777 23522 0 -1 4202496 2521 0 0 0 308 1 0 0 25 0 1 0 528681190 20123648 2254 33554432000 4194304 4785740 140734764014592 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/23779/statm: 4913 2254 340 145 0 1958 0
[pid=23780] ppid=23777 vsize=459160 CPUtime=3.09 cores=2
/proc/23780/stat : 23780 (TNM) R 23777 23777 23522 0 -1 4202496 1425 0 0 0 309 0 0 0 25 0 1 0 528681190 470179840 1322 33554432000 134512640 134534564 4286602496 18446744073709551615 134528392 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/23780/statm: 114790 1322 95 6 0 113456 0
[pid=23781] ppid=23777 vsize=19496 CPUtime=3.09 cores=3
/proc/23781/stat : 23781 (march_hi) R 23777 23777 23522 0 -1 4202496 8054 0 0 0 308 1 0 0 25 0 1 0 528681190 19963904 3017 33554432000 4194304 4331988 140733372199024 18446744073709551615 4210448 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/23781/statm: 4874 3017 175 34 0 3316 0
[pid=23782] ppid=23777 vsize=139036 CPUtime=12.22 cores=4-7
/proc/23782/stat : 23782 (plingeling) S 23777 23777 23522 0 -1 4202496 8584 1259 0 0 1221 1 0 0 19 0 5 0 528681190 142372864 6800 33554432000 4194304 4356188 140737205632000 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23782/statm: 34759 6800 162 40 0 33303 0
[pid=23782/tid=23789] ppid=23777 vsize=139036 CPUtime=3.05 cores=4-7
/proc/23782/task/23789/stat : 23789 (plingeling) R 23777 23777 23522 0 -1 4202560 1480 1259 0 0 305 0 0 0 25 0 5 0 528681195 142372864 6800 33554432000 4194304 4356188 140737205632000 18446744073709551615 4234107 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23782/tid=23790] ppid=23777 vsize=139036 CPUtime=3.05 cores=4-7
/proc/23782/task/23790/stat : 23790 (plingeling) R 23777 23777 23522 0 -1 4202560 1403 1259 0 0 305 0 0 0 25 0 5 0 528681195 142372864 6800 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23782/tid=23791] ppid=23777 vsize=139036 CPUtime=3.04 cores=4-7
/proc/23782/task/23791/stat : 23791 (plingeling) R 23777 23777 23522 0 -1 4202560 1061 1259 0 0 304 0 0 0 25 0 5 0 528681195 142372864 6800 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331019 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23782/tid=23792] ppid=23777 vsize=139036 CPUtime=3.03 cores=4-7
/proc/23782/task/23792/stat : 23792 (plingeling) R 23777 23777 23522 0 -1 4202560 1869 1259 0 0 303 0 0 0 25 0 5 0 528681195 142372864 6800 33554432000 4194304 4356188 140737205632000 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.58
Current children cumulated vsize (KiB) 2299128

[startup+6.30079 s]
/proc/loadavg: 7.42 7.89 7.96 9/180 23792
/proc/meminfo: memFree=30722956/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 6 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
[pid=23778] ppid=23777 vsize=1655376 CPUtime=6.29 cores=0
/proc/23778/stat : 23778 (cryptominisat) R 23777 23777 23522 0 -1 4202496 12276 0 0 0 627 2 0 0 25 0 1 0 528681190 1695105024 6757 33554432000 4194304 4587354 140733422818288 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23778/statm: 413844 6757 337 96 0 410406 0
[pid=23779] ppid=23777 vsize=21896 CPUtime=6.29 cores=1
/proc/23779/stat : 23779 (clasp) R 23777 23777 23522 0 -1 4202496 3051 0 0 0 628 1 0 0 25 0 1 0 528681190 22421504 2784 33554432000 4194304 4785740 140734764014592 18446744073709551615 4287019 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/23779/statm: 5474 2784 340 145 0 2519 0
[pid=23780] ppid=23777 vsize=459160 CPUtime=6.29 cores=2
/proc/23780/stat : 23780 (TNM) R 23777 23777 23522 0 -1 4202496 1425 0 0 0 629 0 0 0 25 0 1 0 528681190 470179840 1322 33554432000 134512640 134534564 4286602496 18446744073709551615 134528288 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/23780/statm: 114790 1322 95 6 0 113456 0
[pid=23781] ppid=23777 vsize=19496 CPUtime=6.29 cores=3
/proc/23781/stat : 23781 (march_hi) R 23777 23777 23522 0 -1 4202496 8054 0 0 0 628 1 0 0 25 0 1 0 528681190 19963904 3017 33554432000 4194304 4331988 140733372199024 18446744073709551615 4239602 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/23781/statm: 4874 3017 175 34 0 3316 0
[pid=23782] ppid=23777 vsize=142108 CPUtime=25 cores=4-7
/proc/23782/stat : 23782 (plingeling) S 23777 23777 23522 0 -1 4202496 10708 1259 0 0 2499 1 0 0 19 0 5 0 528681190 145518592 7568 33554432000 4194304 4356188 140737205632000 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23782/statm: 35527 7568 162 40 0 34071 0
[pid=23782/tid=23789] ppid=23777 vsize=142108 CPUtime=6.24 cores=4-7
/proc/23782/task/23789/stat : 23789 (plingeling) R 23777 23777 23522 0 -1 4202560 2190 1259 0 0 624 0 0 0 25 0 5 0 528681195 145518592 7568 33554432000 4194304 4356188 140737205632000 18446744073709551615 4270872 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23782/tid=23790] ppid=23777 vsize=142108 CPUtime=6.25 cores=4-7
/proc/23782/task/23790/stat : 23790 (plingeling) R 23777 23777 23522 0 -1 4202560 1627 1259 0 0 625 0 0 0 25 0 5 0 528681195 145518592 7568 33554432000 4194304 4356188 140737205632000 18446744073709551615 4271355 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23782/tid=23791] ppid=23777 vsize=142108 CPUtime=6.24 cores=4-7
/proc/23782/task/23791/stat : 23791 (plingeling) R 23777 23777 23522 0 -1 4202560 1768 1259 0 0 624 0 0 0 25 0 5 0 528681195 145518592 7568 33554432000 4194304 4356188 140737205632000 18446744073709551615 4330976 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23782/tid=23792] ppid=23777 vsize=142108 CPUtime=6.21 cores=4-7
/proc/23782/task/23792/stat : 23792 (plingeling) R 23777 23777 23522 0 -1 4202560 2352 1259 0 0 621 0 0 0 25 0 5 0 528681195 145518592 7568 33554432000 4194304 4356188 140737205632000 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.16
Current children cumulated vsize (KiB) 2316360

[startup+12.7007 s]
/proc/loadavg: 7.47 7.89 7.96 9/180 23792
/proc/meminfo: memFree=30699468/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 6 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
[pid=23778] ppid=23777 vsize=1658904 CPUtime=12.69 cores=0
/proc/23778/stat : 23778 (cryptominisat) R 23777 23777 23522 0 -1 4202496 16957 0 0 0 1266 3 0 0 25 0 1 0 528681190 1698717696 9439 33554432000 4194304 4587354 140733422818288 18446744073709551615 4238692 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23778/statm: 414726 9439 337 96 0 411288 0
[pid=23779] ppid=23777 vsize=24876 CPUtime=12.69 cores=1
/proc/23779/stat : 23779 (clasp) R 23777 23777 23522 0 -1 4202496 3761 0 0 0 1268 1 0 0 25 0 1 0 528681190 25473024 3494 33554432000 4194304 4785740 140734764014592 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/23779/statm: 6219 3494 340 145 0 3264 0
[pid=23780] ppid=23777 vsize=459160 CPUtime=12.69 cores=2
/proc/23780/stat : 23780 (TNM) R 23777 23777 23522 0 -1 4202496 1425 0 0 0 1269 0 0 0 25 0 1 0 528681190 470179840 1322 33554432000 134512640 134534564 4286602496 18446744073709551615 134530158 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/23780/statm: 114790 1322 95 6 0 113456 0
[pid=23781] ppid=23777 vsize=19496 CPUtime=12.69 cores=3
/proc/23781/stat : 23781 (march_hi) R 23777 23777 23522 0 -1 4202496 8054 0 0 0 1268 1 0 0 25 0 1 0 528681190 19963904 3017 33554432000 4194304 4331988 140733372199024 18446744073709551615 4210448 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/23781/statm: 4874 3017 175 34 0 3316 0
[pid=23782] ppid=23777 vsize=147252 CPUtime=50.56 cores=4-7
/proc/23782/stat : 23782 (plingeling) S 23777 23777 23522 0 -1 4202496 12672 1259 0 0 5055 1 0 0 19 0 5 0 528681190 150786048 8854 33554432000 4194304 4356188 140737205632000 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23782/statm: 36813 8854 162 40 0 35357 0
[pid=23782/tid=23789] ppid=23777 vsize=147252 CPUtime=12.64 cores=4-7
/proc/23782/task/23789/stat : 23789 (plingeling) R 23777 23777 23522 0 -1 4202560 2574 1259 0 0 1264 0 0 0 25 0 5 0 528681195 150786048 8854 33554432000 4194304 4356188 140737205632000 18446744073709551615 4330973 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23782/tid=23790] ppid=23777 vsize=147252 CPUtime=12.64 cores=4-7
/proc/23782/task/23790/stat : 23790 (plingeling) R 23777 23777 23522 0 -1 4202560 2494 1259 0 0 1264 0 0 0 25 0 5 0 528681195 150786048 8854 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331019 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23782/tid=23791] ppid=23777 vsize=147252 CPUtime=12.64 cores=4-7
/proc/23782/task/23791/stat : 23791 (plingeling) R 23777 23777 23522 0 -1 4202560 2155 1259 0 0 1264 0 0 0 25 0 5 0 528681195 150786048 8854 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331035 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23782/tid=23792] ppid=23777 vsize=147252 CPUtime=12.57 cores=4-7
/proc/23782/task/23792/stat : 23792 (plingeling) R 23777 23777 23522 0 -1 4202560 2678 1259 0 0 1257 0 0 0 25 0 5 0 528681195 150786048 8854 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331076 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.32
Current children cumulated vsize (KiB) 2328012

[startup+25.5006 s]
/proc/loadavg: 7.59 7.89 7.96 9/180 23792
/proc/meminfo: memFree=30681184/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 6 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
[pid=23778] ppid=23777 vsize=1672892 CPUtime=25.49 cores=0
/proc/23778/stat : 23778 (cryptominisat) R 23777 23777 23522 0 -1 4202496 30351 0 0 0 2543 6 0 0 25 0 1 0 528681190 1713041408 12172 33554432000 4194304 4587354 140733422818288 18446744073709551615 4238532 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=23780] ppid=23777 vsize=459160 CPUtime=4782.36 cores=2
/proc/23780/stat : 23780 (TNM) R 23777 23777 23522 0 -1 4202496 1425 0 0 0 478233 3 0 0 25 0 1 0 528681190 470179840 1322 33554432000 134512640 134534564 4286602496 18446744073709551615 134530549 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/23780/statm: 114790 1322 95 6 0 113456 0
[pid=23781] ppid=23777 vsize=87104 CPUtime=4781.56 cores=3
/proc/23781/stat : 23781 (march_hi) R 23777 23777 23522 0 -1 4202496 21414 0 0 0 478143 13 0 0 25 0 1 0 528681190 89194496 15873 33554432000 4194304 4331988 140733372199024 18446744073709551615 4210516 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/23781/statm: 21776 15873 175 34 0 20218 0
[pid=23782] ppid=23777 vsize=356008 CPUtime=19102.8 cores=4-7
/proc/23782/stat : 23782 (plingeling) S 23777 23777 23522 0 -1 4202496 1546849 1259 0 0 1910052 227 0 0 19 0 5 0 528681190 364552192 61075 33554432000 4194304 4356188 140737205632000 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23782/statm: 89002 61075 162 40 0 87546 0
[pid=23782/tid=23789] ppid=23777 vsize=356008 CPUtime=4782.32 cores=4-7
/proc/23782/task/23789/stat : 23789 (plingeling) R 23777 23777 23522 0 -1 4202560 397175 1259 0 0 478175 57 0 0 25 0 5 0 528681195 364552192 61075 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23782/tid=23790] ppid=23777 vsize=356008 CPUtime=4782.32 cores=4-7
/proc/23782/task/23790/stat : 23790 (plingeling) R 23777 23777 23522 0 -1 4202560 454094 1259 0 0 478162 70 0 0 25 0 5 0 528681195 364552192 61075 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23782/tid=23791] ppid=23777 vsize=356008 CPUtime=4781.85 cores=4-7
/proc/23782/task/23791/stat : 23791 (plingeling) R 23777 23777 23522 0 -1 4202560 288249 1259 0 0 478144 41 0 0 25 0 5 0 528681195 364552192 61075 33554432000 4194304 4356188 140737205632000 18446744073709551615 4210704 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23782/tid=23792] ppid=23777 vsize=356008 CPUtime=4756.22 cores=4-7
/proc/23782/task/23792/stat : 23792 (plingeling) R 23777 23777 23522 0 -1 4202560 404560 1259 0 0 475566 56 0 0 25 0 5 0 528681195 364552192 61075 33554432000 4194304 4356188 140737205632000 18446744073709551615 4210615 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38231.3
Current children cumulated vsize (KiB) 2832056

[startup+4842.3 s]
/proc/loadavg: 8.01 8.02 8.00 9/180 23942
/proc/meminfo: memFree=30235552/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 0 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
[pid=23778] ppid=23777 vsize=1775728 CPUtime=4842.22 cores=0
/proc/23778/stat : 23778 (cryptominisat) R 23777 23777 23522 0 -1 4202496 959353 0 0 0 483866 356 0 0 25 0 1 0 528681190 1818345472 30535 33554432000 4194304 4587354 140733422818288 18446744073709551615 4238421 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23778/statm: 443932 30535 337 96 0 440494 0
[pid=23779] ppid=23777 vsize=135784 CPUtime=4842.35 cores=1
/proc/23779/stat : 23779 (clasp) R 23777 23777 23522 0 -1 4202496 29562 0 0 0 484228 7 0 0 25 0 1 0 528681190 139042816 28681 33554432000 4194304 4785740 140734764014592 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/23779/statm: 33946 28681 341 145 0 30991 0
[pid=23780] ppid=23777 vsize=459160 CPUtime=4842.36 cores=2
/proc/23780/stat : 23780 (TNM) R 23777 23777 23522 0 -1 4202496 1425 0 0 0 484233 3 0 0 25 0 1 0 528681190 470179840 1322 33554432000 134512640 134534564 4286602496 18446744073709551615 134528993 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/23780/statm: 114790 1322 95 6 0 113456 0
[pid=23781] ppid=23777 vsize=87104 CPUtime=4841.55 cores=3
/proc/23781/stat : 23781 (march_hi) R 23777 23777 23522 0 -1 4202496 21571 0 0 0 484142 13 0 0 25 0 1 0 528681190 89194496 16030 33554432000 4194304 4331988 140733372199024 18446744073709551615 4239602 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/23781/statm: 21776 16030 175 34 0 20218 0
[pid=23782] ppid=23777 vsize=368296 CPUtime=19342.5 cores=4-7
/proc/23782/stat : 23782 (plingeling) S 23777 23777 23522 0 -1 4202496 1561697 1259 0 0 1934017 229 0 0 19 0 5 0 528681190 377135104 64147 33554432000 4194304 4356188 140737205632000 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23782/statm: 92074 64147 162 40 0 90618 0
[pid=23782/tid=23789] ppid=23777 vsize=368296 CPUtime=4842.32 cores=4-7
/proc/23782/task/23789/stat : 23789 (plingeling) R 23777 23777 23522 0 -1 4202560 401527 1259 0 0 484174 58 0 0 25 0 5 0 528681195 377135104 64147 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23782/tid=23790] ppid=23777 vsize=368296 CPUtime=4842.32 cores=4-7
/proc/23782/task/23790/stat : 23790 (plingeling) R 23777 23777 23522 0 -1 4202560 458574 1259 0 0 484162 70 0 0 25 0 5 0 528681195 377135104 64147 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331045 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23782/tid=23791] ppid=23777 vsize=368296 CPUtime=4841.86 cores=4-7
/proc/23782/task/23791/stat : 23791 (plingeling) R 23777 23777 23522 0 -1 4202560 289785 1259 0 0 484144 42 0 0 25 0 5 0 528681195 377135104 64147 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23782/tid=23792] ppid=23777 vsize=368296 CPUtime=4815.89 cores=4-7
/proc/23782/task/23792/stat : 23792 (plingeling) R 23777 23777 23522 0 -1 4202560 409040 1259 0 0 481532 57 0 0 25 0 5 0 528681195 377135104 64147 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331379 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38710.9
Current children cumulated vsize (KiB) 2844396

[startup+4902.3 s]
/proc/loadavg: 8.07 8.03 8.00 9/180 23943
/proc/meminfo: memFree=30240172/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 0 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
[pid=23778] ppid=23777 vsize=1776476 CPUtime=4902.22 cores=0
/proc/23778/stat : 23778 (cryptominisat) R 23777 23777 23522 0 -1 4202496 970348 0 0 0 489861 361 0 0 25 0 1 0 528681190 1819111424 30448 33554432000 4194304 4587354 140733422818288 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23778/statm: 444119 30448 337 96 0 440681 0
[pid=23779] ppid=23777 vsize=135784 CPUtime=4902.35 cores=1
/proc/23779/stat : 23779 (clasp) R 23777 23777 23522 0 -1 4202496 29568 0 0 0 490228 7 0 0 25 0 1 0 528681190 139042816 28687 33554432000 4194304 4785740 140734764014592 18446744073709551615 4287076 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/23779/statm: 33946 28687 341 145 0 30991 0
[pid=23780] ppid=23777 vsize=459160 CPUtime=4902.36 cores=2
/proc/23780/stat : 23780 (TNM) R 23777 23777 23522 0 -1 4202496 1425 0 0 0 490233 3 0 0 25 0 1 0 528681190 470179840 1322 33554432000 134512640 134534564 4286602496 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/23780/statm: 114790 1322 95 6 0 113456 0
[pid=23781] ppid=23777 vsize=87104 CPUtime=4901.55 cores=3
/proc/23781/stat : 23781 (march_hi) R 23777 23777 23522 0 -1 4202496 21729 0 0 0 490142 13 0 0 25 0 1 0 528681190 89194496 16188 33554432000 4194304 4331988 140733372199024 18446744073709551615 4226263 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/23781/statm: 21776 16188 175 34 0 20218 0
[pid=23782] ppid=23777 vsize=365224 CPUtime=19582.1 cores=4-7
/proc/23782/stat : 23782 (plingeling) S 23777 23777 23522 0 -1 4202496 1575009 1259 0 0 1957983 230 0 0 19 0 5 0 528681190 373989376 63379 33554432000 4194304 4356188 140737205632000 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23782/statm: 91306 63379 162 40 0 89850 0
[pid=23782/tid=23789] ppid=23777 vsize=365224 CPUtime=4902.32 cores=4-7
/proc/23782/task/23789/stat : 23789 (plingeling) R 23777 23777 23522 0 -1 4202560 404343 1259 0 0 490174 58 0 0 25 0 5 0 528681195 373989376 63379 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23782/tid=23790] ppid=23777 vsize=365224 CPUtime=4902.32 cores=4-7
/proc/23782/task/23790/stat : 23790 (plingeling) R 23777 23777 23522 0 -1 4202560 465614 1259 0 0 490161 71 0 0 25 0 5 0 528681195 373989376 63379 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331964 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23782/tid=23791] ppid=23777 vsize=365224 CPUtime=4901.85 cores=4-7
/proc/23782/task/23791/stat : 23791 (plingeling) R 23777 23777 23522 0 -1 4202560 290297 1259 0 0 490143 42 0 0 25 0 5 0 528681195 373989376 63379 33554432000 4194304 4356188 140737205632000 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23782/tid=23792] ppid=23777 vsize=365224 CPUtime=4875.57 cores=4-7
/proc/23782/task/23792/stat : 23792 (plingeling) R 23777 23777 23522 0 -1 4202560 411984 1259 0 0 487499 58 0 0 25 0 5 0 528681195 373989376 63379 33554432000 4194304 4356188 140737205632000 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39190.6
Current children cumulated vsize (KiB) 2842072

[startup+4962.3 s]
/proc/loadavg: 8.09 8.04 8.00 9/180 23945
/proc/meminfo: memFree=30242140/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 0 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
[pid=23778] ppid=23777 vsize=1776564 CPUtime=4962.22 cores=0
/proc/23778/stat : 23778 (cryptominisat) R 23777 23777 23522 0 -1 4202496 981387 0 0 0 495856 366 0 0 25 0 1 0 528681190 1819201536 30229 33554432000 4194304 4587354 140733422818288 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23778/statm: 444141 30229 337 96 0 440703 0
[pid=23779] ppid=23777 vsize=135784 CPUtime=4962.35 cores=1
/proc/23779/stat : 23779 (clasp) R 23777 23777 23522 0 -1 4202496 29568 0 0 0 496228 7 0 0 25 0 1 0 528681190 139042816 28687 33554432000 4194304 4785740 140734764014592 18446744073709551615 4287088 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/23779/statm: 33946 28687 341 145 0 30991 0
[pid=23780] ppid=23777 vsize=459160 CPUtime=4962.36 cores=2
/proc/23780/stat : 23780 (TNM) R 23777 23777 23522 0 -1 4202496 1425 0 0 0 496233 3 0 0 25 0 1 0 528681190 470179840 1322 33554432000 134512640 134534564 4286602496 18446744073709551615 134530549 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/23780/statm: 114790 1322 95 6 0 113456 0
[pid=23781] ppid=23777 vsize=87104 CPUtime=4961.55 cores=3
/proc/23781/stat : 23781 (march_hi) R 23777 23777 23522 0 -1 4202496 21890 0 0 0 496142 13 0 0 25 0 1 0 528681190 89194496 16349 33554432000 4194304 4331988 140733372199024 18446744073709551615 4210430 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/23781/statm: 21776 16349 175 34 0 20218 0
[pid=23782] ppid=23777 vsize=363176 CPUtime=19821.8 cores=4-7
/proc/23782/stat : 23782 (plingeling) S 23777 23777 23522 0 -1 4202496 1596513 1259 0 0 1981946 233 0 0 19 0 5 0 528681190 371892224 62867 33554432000 4194304 4356188 140737205632000 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23782/statm: 90794 62867 162 40 0 89338 0
[pid=23782/tid=23789] ppid=23777 vsize=363176 CPUtime=4962.32 cores=4-7
/proc/23782/task/23789/stat : 23789 (plingeling) R 23777 23777 23522 0 -1 4202560 412791 1259 0 0 496173 59 0 0 25 0 5 0 528681195 371892224 62867 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331028 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23782/tid=23790] ppid=23777 vsize=363176 CPUtime=4962.32 cores=4-7
/proc/23782/task/23790/stat : 23790 (plingeling) R 23777 23777 23522 0 -1 4202560 467534 1259 0 0 496161 71 0 0 25 0 5 0 528681195 371892224 62867 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23782/tid=23791] ppid=23777 vsize=363176 CPUtime=4961.84 cores=4-7
/proc/23782/task/23791/stat : 23791 (plingeling) R 23777 23777 23522 0 -1 4202560 297977 1259 0 0 496141 43 0 0 25 0 5 0 528681195 371892224 62867 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23782/tid=23792] ppid=23777 vsize=363176 CPUtime=4935.24 cores=4-7
/proc/23782/task/23792/stat : 23792 (plingeling) R 23777 23777 23522 0 -1 4202560 415440 1259 0 0 493466 58 0 0 25 0 5 0 528681195 371892224 62867 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331960 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39670.3
Current children cumulated vsize (KiB) 2840112



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.6 s]
/proc/loadavg: 8.11 8.05 8.00 9/180 23947
/proc/meminfo: memFree=30199724/32950928 swapFree=67111448/67111528
[pid=23777] ppid=23775 vsize=18324 CPUtime=0 cores=0-7
/proc/23777/stat : 23777 (ppfolio) S 23775 23777 23522 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 528681190 18763776 389 33554432000 4194304 4287625 140733498495792 18446744073709551615 251814260239 0 0 4096 65536 12306878459090567167 0 0 17 0 0 0 0
/proc/23777/statm: 4581 389 341 23 0 91 0
[pid=23778] ppid=23777 vsize=1776564 CPUtime=5003.52 cores=0
/proc/23778/stat : 23778 (cryptominisat) R 23777 23777 23522 0 -1 4202496 986155 0 0 0 499984 368 0 0 25 0 1 0 528681190 1819201536 34878 33554432000 4194304 4587354 140733422818288 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/23778/statm: 444141 34878 337 96 0 440703 0
[pid=23779] ppid=23777 vsize=136852 CPUtime=5003.65 cores=1
/proc/23779/stat : 23779 (clasp) R 23777 23777 23522 0 -1 4202496 29811 0 0 0 500358 7 0 0 25 0 1 0 528681190 140136448 28930 33554432000 4194304 4785740 140734764014592 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/23779/statm: 34213 28930 341 145 0 31258 0
[pid=23780] ppid=23777 vsize=459160 CPUtime=5003.67 cores=2
/proc/23780/stat : 23780 (TNM) R 23777 23777 23522 0 -1 4202496 1425 0 0 0 500364 3 0 0 25 0 1 0 528681190 470179840 1322 33554432000 134512640 134534564 4286602496 18446744073709551615 134528737 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/23780/statm: 114790 1322 95 6 0 113456 0
[pid=23781] ppid=23777 vsize=87104 CPUtime=5002.86 cores=3
/proc/23781/stat : 23781 (march_hi) R 23777 23777 23522 0 -1 4202496 22005 0 0 0 500273 13 0 0 25 0 1 0 528681190 89194496 16464 33554432000 4194304 4331988 140733372199024 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/23781/statm: 21776 16464 175 34 0 20218 0
[pid=23782] ppid=23777 vsize=386216 CPUtime=19986.8 cores=4-7
/proc/23782/stat : 23782 (plingeling) S 23777 23777 23522 0 -1 4202496 1610081 1259 0 0 1998444 235 0 0 19 0 5 0 528681190 395485184 68627 33554432000 4194304 4356188 140737205632000 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23782/statm: 96554 68627 162 40 0 95098 0
[pid=23782/tid=23789] ppid=23777 vsize=386216 CPUtime=5003.62 cores=4-7
/proc/23782/task/23789/stat : 23789 (plingeling) R 23777 23777 23522 0 -1 4202560 416631 1259 0 0 500303 59 0 0 25 0 5 0 528681195 395485184 68627 33554432000 4194304 4356188 140737205632000 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23782/tid=23790] ppid=23777 vsize=386216 CPUtime=5003.62 cores=4-7
/proc/23782/task/23790/stat : 23790 (plingeling) R 23777 23777 23522 0 -1 4202560 475214 1259 0 0 500290 72 0 0 25 0 5 0 528681195 395485184 68627 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23782/tid=23791] ppid=23777 vsize=386216 CPUtime=5003.15 cores=4-7
/proc/23782/task/23791/stat : 23791 (plingeling) R 23777 23777 23522 0 -1 4202560 297977 1259 0 0 500272 43 0 0 25 0 5 0 528681195 395485184 68627 33554432000 4194304 4356188 140737205632000 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23782/tid=23792] ppid=23777 vsize=386216 CPUtime=4976.32 cores=4-7
/proc/23782/task/23792/stat : 23792 (plingeling) R 23777 23777 23522 0 -1 4202560 417488 1259 0 0 497573 59 0 0 25 0 5 0 528681195 395485184 68627 33554432000 4194304 4356188 140737205632000 18446744073709551615 4330992 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 2864220

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 23777 and gives
#  childrusage.ru_utime.tv_sec=20009
#  childrusage.ru_utime.tv_usec=812046
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=958398
# CPU time returned by wait4() is 20013.8
# while last known CPU time is 40000.5
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5003.62
CPU time (s): 40000.5
CPU user time (s): 39994.2
CPU system time (s): 6.26
CPU usage (%): 799.431
Max. virtual memory (cumulated for all children) (KiB): 11530976

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 20009.8
system time used= 3.9584
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1039931
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 890
involuntary context switches= 81910

runsolver used 7.59284 second user time and 19.697 second system time

The end

Launcher Data 

Begin job on node126 at 2011-04-30 15:40:25
IDJOB=3275801
IDBENCH=83336
IDSOLVER=1638
FILE ID=node126/3275801-1304170825
RUNJOBID= node126-1304165815-23536
PBS_JOBID= 13170951
Free space on /tmp= 73184 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden3k/VanDerWaerden_2-3-19_349.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3275801-1304170825/watcher-3275801-1304170825 -o /tmp/evaluation-result-3275801-1304170825/solver-3275801-1304170825 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3275801-1304170825.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= faed4f95ee485d18812f5d39a28df3e5
RANDOM SEED=514436750

node126.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30798880 kB
Buffers:        296904 kB
Cached:        1478696 kB
SwapCached:         76 kB
Active:         205920 kB
Inactive:      1634076 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30798880 kB
SwapTotal:    67111528 kB
SwapFree:     67111448 kB
Dirty:           10484 kB
Writeback:           0 kB
AnonPages:       64240 kB
Mapped:          14712 kB
Slab:           247688 kB
PageTables:       4244 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   184740 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73172 MiB
End job on node126 at 2011-04-30 17:03:51