Trace number 3310244

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 30447.2 5100.02

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.030-NOTKNOWN.cnf
MD5SUMc6d3480970bb2819448dd0a800a3cf1c
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark3527.13
Satisfiable
(Un)Satisfiability was proved
Number of variables1134832
Number of clauses6175026
Sum of the clauses size14647926
Maximum clause length1178
Minimum clause length1
Number of clauses of size 1846
Number of clauses of size 26124260
Number of clauses of size 3960
Number of clauses of size 41200
Number of clauses of size 51200
Number of clauses of size over 546560

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7
0.00/0.01

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3310244-1304007440/watcher-3310244-1304007440 -o /tmp/evaluation-result-3310244-1304007440/solver-3310244-1304007440 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3310244-1304007440.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.28 1.81 1.87 2/168 17325
/proc/meminfo: memFree=15803928/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=0 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 215 0 0 0 0 0 0 25 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 9224195738567704576 0 0 17 2 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0

[startup+0.0759619 s]
/proc/loadavg: 1.28 1.81 1.87 2/168 17325
/proc/meminfo: memFree=15803928/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=0 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 215 0 0 0 0 0 0 25 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 9224195738567704576 0 0 17 2 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.10096 s]
/proc/loadavg: 1.28 1.81 1.87 2/168 17325
/proc/meminfo: memFree=15803928/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=0 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 215 0 0 0 0 0 0 25 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 9224195738567704576 0 0 17 2 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300925 s]
/proc/loadavg: 1.28 1.81 1.87 2/168 17325
/proc/meminfo: memFree=15803928/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=0 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 215 0 0 0 0 0 0 25 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 9224195738567704576 0 0 17 2 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700854 s]
/proc/loadavg: 1.28 1.81 1.87 2/168 17325
/proc/meminfo: memFree=15803928/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=0 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 215 0 0 0 0 0 0 25 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 9224195738567704576 0 0 17 2 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50071 s]
/proc/loadavg: 1.50 1.85 1.88 5/173 17336
/proc/meminfo: memFree=15106024/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=0 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 215 0 0 0 0 0 0 25 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 9224195738567704576 0 0 17 2 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
[pid=17326] ppid=17325 vsize=346664 CPUtime=1.49 cores=0
/proc/17326/stat : 17326 (cryptominisat) R 17325 17325 17299 0 -1 4202496 77188 0 0 0 136 13 0 0 20 0 1 0 512344013 354983936 76029 33554432000 4194304 4587354 140733652513184 18446744073709551615 274664469189 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17326/statm: 86666 76030 288 96 0 83228 0
[pid=17327] ppid=17325 vsize=235092 CPUtime=1.48 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 62675 0 0 0 138 10 0 0 20 0 1 0 512344013 240734208 54253 33554432000 4194304 4785740 140733201668592 18446744073709551615 274664808192 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 58773 54253 309 145 0 55818 0
[pid=17329] ppid=17325 vsize=277240 CPUtime=1.48 cores=3
/proc/17329/stat : 17329 (march_hi) R 17325 17325 17299 0 -1 4194304 63289 0 0 0 140 8 0 0 20 0 1 0 512344013 283893760 63184 33554432000 4194304 4331988 140736434608752 18446744073709551615 274664469189 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17329/statm: 69310 63184 113 34 0 67752 0
[pid=17330] ppid=17325 vsize=212312 CPUtime=1.48 cores=4-7
/proc/17330/stat : 17330 (plingeling) R 17325 17325 17299 0 -1 4202496 51965 1250 0 0 141 7 0 0 21 0 1 0 512344013 217407488 51741 33554432000 4194304 4356188 140735304150512 18446744073709551615 274664808192 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/17330/statm: 53078 51741 132 40 0 51622 0
Current children cumulated CPU time (s) 5.93
Current children cumulated vsize (KiB) 1089632

[startup+34.073 s]
/proc/loadavg: 1.50 1.85 1.88 5/173 17336
/proc/meminfo: memFree=15106024/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=0 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 215 0 0 0 0 0 0 15 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 9224195738567704576 0 0 17 2 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
[pid=17326] ppid=17325 vsize=1067436 CPUtime=34.06 cores=0
/proc/17326/stat : 17326 (cryptominisat) R 17325 17325 17299 0 -1 4202496 348910 0 0 0 3357 49 0 0 25 0 1 0 512344013 1093054464 241486 33554432000 4194304 4587354 140733652513184 18446744073709551615 4236860 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17326/statm: 266859 241486 330 96 0 263421 0
[pid=17327] ppid=17325 vsize=281960 CPUtime=12.21 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 97661 0 0 0 1053 168 0 0 25 0 1 0 512344013 288727040 66796 33554432000 4194304 4785740 140733201668592 18446744073709551615 4544143 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 70490 66796 336 145 0 67535 0
[pid=17329] ppid=17325 vsize=795124 CPUtime=34.05 cores=3
/proc/17329/stat : 17329 (march_hi) R 17325 17325 17299 0 -1 4194304 734207 0 0 0 3306 99 0 0 25 0 1 0 512344013 814206976 137041 33554432000 4194304 4331988 140736434608752 18446744073709551615 4256793 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17329/statm: 198781 137041 129 34 0 197223 0
[pid=17330] ppid=17325 vsize=1048048 CPUtime=115.73 cores=4-7
/proc/17330/stat : 17330 (plingeling) S 17325 17325 17299 0 -1 4202496 478535 1250 0 0 11447 126 0 0 25 0 5 0 512344013 1073201152 241542 33554432000 4194304 4356188 140735304150512 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17330/statm: 262012 241542 164 40 0 260556 0
Current children cumulated CPU time (s) 196.05
Current children cumulated vsize (KiB) 3210892

[startup+34.101 s]
/proc/loadavg: 1.50 1.85 1.88 5/173 17336
/proc/meminfo: memFree=15106024/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=0 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 215 0 0 0 0 0 0 15 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 9224195738567704576 0 0 17 2 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
[pid=17326] ppid=17325 vsize=1067436 CPUtime=34.09 cores=0
/proc/17326/stat : 17326 (cryptominisat) R 17325 17325 17299 0 -1 4202496 348910 0 0 0 3360 49 0 0 25 0 1 0 512344013 1093054464 241486 33554432000 4194304 4587354 140733652513184 18446744073709551615 4236561 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17326/statm: 266859 241486 330 96 0 263421 0
[pid=17327] ppid=17325 vsize=281960 CPUtime=12.22 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 97661 0 0 0 1054 168 0 0 25 0 1 0 512344013 288727040 66796 33554432000 4194304 4785740 140733201668592 18446744073709551615 4561965 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 70490 66796 336 145 0 67535 0
[pid=17329] ppid=17325 vsize=795124 CPUtime=34.08 cores=3
/proc/17329/stat : 17329 (march_hi) R 17325 17325 17299 0 -1 4194304 734207 0 0 0 3309 99 0 0 25 0 1 0 512344013 814206976 137041 33554432000 4194304 4331988 140736434608752 18446744073709551615 4256793 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17329/statm: 198781 137041 129 34 0 197223 0
[pid=17330] ppid=17325 vsize=1048048 CPUtime=115.85 cores=4-7
/proc/17330/stat : 17330 (plingeling) S 17325 17325 17299 0 -1 4202496 478535 1250 0 0 11459 126 0 0 25 0 5 0 512344013 1073201152 241542 33554432000 4194304 4356188 140735304150512 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17330/statm: 262012 241542 164 40 0 260556 0
Current children cumulated CPU time (s) 196.24
Current children cumulated vsize (KiB) 3210892

[startup+34.201 s]
/proc/loadavg: 1.50 1.85 1.88 5/173 17336
/proc/meminfo: memFree=15106024/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=0 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 215 0 0 0 0 0 0 15 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 9224195738567704576 0 0 17 2 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
[pid=17326] ppid=17325 vsize=1067436 CPUtime=34.19 cores=0
/proc/17326/stat : 17326 (cryptominisat) R 17325 17325 17299 0 -1 4202496 348910 0 0 0 3370 49 0 0 25 0 1 0 512344013 1093054464 241486 33554432000 4194304 4587354 140733652513184 18446744073709551615 4236798 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17326/statm: 266859 241486 330 96 0 263421 0
[pid=17327] ppid=17325 vsize=281960 CPUtime=12.26 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 97661 0 0 0 1058 168 0 0 25 0 1 0 512344013 288727040 66796 33554432000 4194304 4785740 140733201668592 18446744073709551615 4544151 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 70490 66796 336 145 0 67535 0
[pid=17329] ppid=17325 vsize=795124 CPUtime=34.18 cores=3
/proc/17329/stat : 17329 (march_hi) R 17325 17325 17299 0 -1 4194304 734207 0 0 0 3319 99 0 0 25 0 1 0 512344013 814206976 137041 33554432000 4194304 4331988 140736434608752 18446744073709551615 4256838 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17329/statm: 198781 137041 129 34 0 197223 0
[pid=17330] ppid=17325 vsize=1048048 CPUtime=116.25 cores=4-7
/proc/17330/stat : 17330 (plingeling) S 17325 17325 17299 0 -1 4202496 478535 1250 0 0 11499 126 0 0 25 0 5 0 512344013 1073201152 241542 33554432000 4194304 4356188 140735304150512 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17330/statm: 262012 241542 164 40 0 260556 0
Current children cumulated CPU time (s) 196.88
Current children cumulated vsize (KiB) 3210892

[startup+34.301 s]
/proc/loadavg: 1.50 1.85 1.88 5/173 17336
/proc/meminfo: memFree=15106024/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=0 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 215 0 0 0 0 0 0 15 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 9224195738567704576 0 0 17 2 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
[pid=17326] ppid=17325 vsize=1067436 CPUtime=34.29 cores=0
/proc/17326/stat : 17326 (cryptominisat) R 17325 17325 17299 0 -1 4202496 348910 0 0 0 3380 49 0 0 25 0 1 0 512344013 1093054464 241486 33554432000 4194304 4587354 140733652513184 18446744073709551615 4236832 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17326/statm: 266859 241486 330 96 0 263421 0
[pid=17327] ppid=17325 vsize=281960 CPUtime=12.31 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 97661 0 0 0 1062 169 0 0 25 0 1 0 512344013 288727040 66796 33554432000 4194304 4785740 140733201668592 18446744073709551615 4561956 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 70490 66796 336 145 0 67535 0
[pid=17329] ppid=17325 vsize=795124 CPUtime=34.28 cores=3
/proc/17329/stat : 17329 (march_hi) R 17325 17325 17299 0 -1 4194304 734207 0 0 0 3329 99 0 0 25 0 1 0 512344013 814206976 137041 33554432000 4194304 4331988 140736434608752 18446744073709551615 4256854 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17329/statm: 198781 137041 129 34 0 197223 0
[pid=17330] ppid=17325 vsize=1048048 CPUtime=116.65 cores=4-7
/proc/17330/stat : 17330 (plingeling) S 17325 17325 17299 0 -1 4202496 478535 1250 0 0 11539 126 0 0 25 0 5 0 512344013 1073201152 241542 33554432000 4194304 4356188 140735304150512 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17330/statm: 262012 241542 164 40 0 260556 0
Current children cumulated CPU time (s) 197.53
Current children cumulated vsize (KiB) 3210892

[startup+51.1071 s]
/proc/loadavg: 5.97 3.05 2.29 8/177 17342
/proc/meminfo: memFree=13023584/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=0 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 215 0 0 0 0 0 0 15 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 9224195742862639364 0 0 17 2 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
[pid=17326] ppid=17325 vsize=1217440 CPUtime=51.1 cores=0
/proc/17326/stat : 17326 (cryptominisat) R 17325 17325 17299 0 -1 4202496 351196 0 0 0 5060 50 0 0 25 0 1 0 512344013 1246658560 243769 33554432000 4194304 4587354 140733652513184 18446744073709551615 4236573 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17326/statm: 304360 243769 330 96 0 300922 0
[pid=17327] ppid=17325 vsize=328584 CPUtime=26.57 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 109333 0 0 0 2474 183 0 0 25 0 1 0 512344013 336470016 78468 33554432000 4194304 4785740 140733201668592 18446744073709551615 4290592 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 82146 78468 336 145 0 79191 0
[pid=17329] ppid=17325 vsize=884424 CPUtime=51.08 cores=3
/proc/17329/stat : 17329 (march_hi) R 17325 17325 17299 0 -1 4194304 1054948 0 0 0 4967 141 0 0 25 0 1 0 512344013 905650176 156499 33554432000 4194304 4331988 140736434608752 18446744073709551615 274664469189 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/17329/statm: 221106 156503 129 34 0 219548 0
[pid=17330] ppid=17325 vsize=1048028 CPUtime=183.77 cores=4-7
/proc/17330/stat : 17330 (plingeling) S 17325 17325 17299 0 -1 4202496 478535 1250 0 0 18251 126 0 0 25 0 5 0 512344013 1073180672 241537 33554432000 4194304 4356188 140735304150512 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17330/statm: 262007 241537 164 40 0 260551 0
[pid=17330/tid=17338] ppid=17325 vsize=1048028 CPUtime=44.76 cores=4-7
/proc/17330/task/17338/stat : 17338 (plingeling) R 17325 17325 17299 0 -1 4202560 82872 1250 0 0 4455 21 0 0 25 0 5 0 512344628 1073180672 241537 33554432000 4194304 4356188 140735304150512 18446744073709551615 4210575 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17330/tid=17339] ppid=17325 vsize=1048028 CPUtime=43.41 cores=4-7
/proc/17330/task/17339/stat : 17339 (plingeling) R 17325 17325 17299 0 -1 4202560 50622 1250 0 0 4320 21 0 0 25 0 5 0 512344628 1073180672 241537 33554432000 4194304 4356188 140735304150512 18446744073709551615 4312072 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17330/tid=17340] ppid=17325 vsize=1048028 CPUtime=44.8 cores=4-7
/proc/17330/task/17340/stat : 17340 (plingeling) R 17325 17325 17299 0 -1 4202560 73806 1250 0 0 4456 24 0 0 25 0 5 0 512344628 1073180672 241537 33554432000 4194304 4356188 140735304150512 18446744073709551615 4211804 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17330/tid=17341] ppid=17325 vsize=1048028 CPUtime=44.62 cores=4-7
/proc/17330/task/17341/stat : 17341 (plingeling) R 17325 17325 17299 0 -1 4202560 69242 1250 0 0 4431 31 0 0 25 0 5 0 512344628 1073180672 241537 33554432000 4194304 4356188 140735304150512 18446744073709551615 4281677 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 312.52
Current children cumulated vsize (KiB) 3496800

[startup+102.301 s]
/proc/loadavg: 6.55 3.65 2.54 8/177 17343

################
# More data... #
################

/proc/17326/statm: 909681 480537 340 96 0 906243 0
[pid=17327] ppid=17325 vsize=1990548 CPUtime=4534.89 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 524605 0 0 0 448350 5139 0 0 25 0 1 0 512344013 2038321152 493740 33554432000 4194304 4785740 140733201668592 18446744073709551615 4290616 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 497637 493740 336 145 0 494682 0
[pid=17330] ppid=17325 vsize=1592592 CPUtime=19317.7 cores=4-7
/proc/17330/stat : 17330 (plingeling) S 17325 17325 17299 0 -1 4202496 2401799 1250 0 0 1930978 794 0 0 25 0 5 0 512344013 1630814208 372433 33554432000 4194304 4356188 140735304150512 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17330/statm: 398148 372433 164 40 0 396692 0
[pid=17330/tid=17338] ppid=17325 vsize=1592592 CPUtime=4810.84 cores=4-7
/proc/17330/task/17338/stat : 17338 (plingeling) R 17325 17325 17299 0 -1 4202560 821012 1250 0 0 480937 147 0 0 25 0 5 0 512344628 1630814208 372433 33554432000 4194304 4356188 140735304150512 18446744073709551615 4270728 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17330/tid=17339] ppid=17325 vsize=1592592 CPUtime=4831.37 cores=4-7
/proc/17330/task/17339/stat : 17339 (plingeling) R 17325 17325 17299 0 -1 4202560 412899 1250 0 0 482855 282 0 0 25 0 5 0 512344628 1630814208 372433 33554432000 4194304 4356188 140735304150512 18446744073709551615 4274268 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17330/tid=17340] ppid=17325 vsize=1592592 CPUtime=4833.41 cores=4-7
/proc/17330/task/17340/stat : 17340 (plingeling) R 17325 17325 17299 0 -1 4202560 421891 1250 0 0 483101 240 0 0 25 0 5 0 512344628 1630814208 372433 33554432000 4194304 4356188 140735304150512 18446744073709551615 4284253 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17330/tid=17341] ppid=17325 vsize=1592592 CPUtime=4835.91 cores=4-7
/proc/17330/task/17341/stat : 17341 (plingeling) R 17325 17325 17299 0 -1 4202560 544004 1250 0 0 483496 95 0 0 25 0 5 0 512344628 1630814208 372433 33554432000 4194304 4356188 140735304150512 18446744073709551615 4331062 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 28902.4
Current children cumulated vsize (KiB) 7240188

[startup+4902.3 s]
/proc/loadavg: 6.02 6.02 6.00 7/176 17484
/proc/meminfo: memFree=8709232/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=208 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 2527177 0 0 20 43 20189 548 15 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 4606797314852388864 0 0 17 0 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
[pid=17326] ppid=17325 vsize=3638724 CPUtime=4901.75 cores=0
/proc/17326/stat : 17326 (cryptominisat) R 17325 17325 17299 0 -1 4202496 284090676 0 0 0 453958 36217 0 0 25 0 1 0 512344013 3726053376 510376 33554432000 4194304 4587354 140733652513184 18446744073709551615 4236606 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17326/statm: 909681 510376 340 96 0 906243 0
[pid=17327] ppid=17325 vsize=1995564 CPUtime=4594.88 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 525833 0 0 0 454349 5139 0 0 25 0 1 0 512344013 2043457536 494968 33554432000 4194304 4785740 140733201668592 18446744073709551615 4290588 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 498891 494968 336 145 0 495936 0
[pid=17330] ppid=17325 vsize=1592552 CPUtime=19557.4 cores=4-7
/proc/17330/stat : 17330 (plingeling) S 17325 17325 17299 0 -1 4202496 2422280 1250 0 0 1954941 797 0 0 25 0 5 0 512344013 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17330/statm: 398138 372423 164 40 0 396682 0
[pid=17330/tid=17338] ppid=17325 vsize=1592552 CPUtime=4870.51 cores=4-7
/proc/17330/task/17338/stat : 17338 (plingeling) R 17325 17325 17299 0 -1 4202560 821012 1250 0 0 486904 147 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4237079 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17330/tid=17339] ppid=17325 vsize=1592552 CPUtime=4891.36 cores=4-7
/proc/17330/task/17339/stat : 17339 (plingeling) R 17325 17325 17299 0 -1 4202560 429284 1250 0 0 488853 283 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4330990 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17330/tid=17340] ppid=17325 vsize=1592552 CPUtime=4893.4 cores=4-7
/proc/17330/task/17340/stat : 17340 (plingeling) R 17325 17325 17299 0 -1 4202560 421891 1250 0 0 489099 241 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17330/tid=17341] ppid=17325 vsize=1592552 CPUtime=4895.92 cores=4-7
/proc/17330/task/17341/stat : 17341 (plingeling) R 17325 17325 17299 0 -1 4202560 548100 1250 0 0 489496 96 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4330796 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 29262
Current children cumulated vsize (KiB) 7245164

[startup+4962.3 s]
/proc/loadavg: 6.00 6.02 6.00 7/176 17486
/proc/meminfo: memFree=8862988/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=208 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 2527177 0 0 20 43 20189 548 15 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 4606797589730295808 0 0 17 0 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
[pid=17326] ppid=17325 vsize=1992020 CPUtime=4961.75 cores=0
/proc/17326/stat : 17326 (cryptominisat) R 17325 17325 17299 0 -1 4202496 284090812 0 0 0 459957 36218 0 0 25 0 1 0 512344013 2039828480 471979 33554432000 4194304 4587354 140733652513184 18446744073709551615 4237643 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17326/statm: 498005 471979 340 96 0 494567 0
[pid=17327] ppid=17325 vsize=1995564 CPUtime=4654.89 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 525833 0 0 0 460350 5139 0 0 25 0 1 0 512344013 2043457536 494968 33554432000 4194304 4785740 140733201668592 18446744073709551615 4290633 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 498891 494968 336 145 0 495936 0
[pid=17330] ppid=17325 vsize=1592552 CPUtime=19797.1 cores=4-7
/proc/17330/stat : 17330 (plingeling) S 17325 17325 17299 0 -1 4202496 2430472 1250 0 0 1978907 799 0 0 25 0 5 0 512344013 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17330/statm: 398138 372423 164 40 0 396682 0
[pid=17330/tid=17338] ppid=17325 vsize=1592552 CPUtime=4930.18 cores=4-7
/proc/17330/task/17338/stat : 17338 (plingeling) R 17325 17325 17299 0 -1 4202560 825108 1250 0 0 492871 147 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4242074 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17330/tid=17339] ppid=17325 vsize=1592552 CPUtime=4951.37 cores=4-7
/proc/17330/task/17339/stat : 17339 (plingeling) R 17325 17325 17299 0 -1 4202560 429284 1250 0 0 494854 283 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4286843 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17330/tid=17340] ppid=17325 vsize=1592552 CPUtime=4953.4 cores=4-7
/proc/17330/task/17340/stat : 17340 (plingeling) R 17325 17325 17299 0 -1 4202560 425987 1250 0 0 495097 243 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17330/tid=17341] ppid=17325 vsize=1592552 CPUtime=4955.92 cores=4-7
/proc/17330/task/17341/stat : 17341 (plingeling) R 17325 17325 17299 0 -1 4202560 548100 1250 0 0 495496 96 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 29621.7
Current children cumulated vsize (KiB) 5598460

[startup+5022.3 s]
/proc/loadavg: 6.11 6.04 6.01 7/176 17487
/proc/meminfo: memFree=8850956/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=208.01 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 2527177 0 0 20 44 20189 548 15 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 4606797314852388864 0 0 17 0 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
[pid=17326] ppid=17325 vsize=1992020 CPUtime=5021.75 cores=0
/proc/17326/stat : 17326 (cryptominisat) R 17325 17325 17299 0 -1 4202496 284090839 0 0 0 465956 36219 0 0 25 0 1 0 512344013 2039828480 472006 33554432000 4194304 4587354 140733652513184 18446744073709551615 4503128 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17326/statm: 498005 472006 340 96 0 494567 0
[pid=17327] ppid=17325 vsize=2007916 CPUtime=4714.89 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 528947 0 0 0 466348 5141 0 0 25 0 1 0 512344013 2056105984 498082 33554432000 4194304 4785740 140733201668592 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 501979 498082 336 145 0 499024 0
[pid=17330] ppid=17325 vsize=1592552 CPUtime=20036.7 cores=4-7
/proc/17330/stat : 17330 (plingeling) S 17325 17325 17299 0 -1 4202496 2430472 1250 0 0 2002874 799 0 0 25 0 5 0 512344013 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17330/statm: 398138 372423 164 40 0 396682 0
[pid=17330/tid=17338] ppid=17325 vsize=1592552 CPUtime=4989.86 cores=4-7
/proc/17330/task/17338/stat : 17338 (plingeling) R 17325 17325 17299 0 -1 4202560 825108 1250 0 0 498839 147 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4297520 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17330/tid=17339] ppid=17325 vsize=1592552 CPUtime=5011.37 cores=4-7
/proc/17330/task/17339/stat : 17339 (plingeling) R 17325 17325 17299 0 -1 4202560 429284 1250 0 0 500854 283 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4208943 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17330/tid=17340] ppid=17325 vsize=1592552 CPUtime=5013.39 cores=4-7
/proc/17330/task/17340/stat : 17340 (plingeling) R 17325 17325 17299 0 -1 4202560 425987 1250 0 0 501096 243 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4331328 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17330/tid=17341] ppid=17325 vsize=1592552 CPUtime=5015.92 cores=4-7
/proc/17330/task/17341/stat : 17341 (plingeling) R 17325 17325 17299 0 -1 4202560 548100 1250 0 0 501496 96 0 0 25 0 5 0 512344628 1630773248 372423 33554432000 4194304 4356188 140735304150512 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 29981.4
Current children cumulated vsize (KiB) 5610812

[startup+5082.3 s]
/proc/loadavg: 6.04 6.03 6.00 7/176 17489
/proc/meminfo: memFree=8827144/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=208.01 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 2527177 0 0 20 44 20189 548 15 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 4606797039974481920 0 0 17 0 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
[pid=17326] ppid=17325 vsize=1992020 CPUtime=5081.74 cores=0
/proc/17326/stat : 17326 (cryptominisat) R 17325 17325 17299 0 -1 4202496 284090839 0 0 0 471953 36221 0 0 25 0 1 0 512344013 2039828480 472006 33554432000 4194304 4587354 140733652513184 18446744073709551615 4507351 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17326/statm: 498005 472006 340 96 0 494567 0
[pid=17327] ppid=17325 vsize=2023612 CPUtime=4774.9 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 532864 0 0 0 472348 5142 0 0 25 0 1 0 512344013 2072178688 501999 33554432000 4194304 4785740 140733201668592 18446744073709551615 4290592 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 505903 501999 336 145 0 502948 0
[pid=17330] ppid=17325 vsize=1600740 CPUtime=20276.4 cores=4-7
/proc/17330/stat : 17330 (plingeling) S 17325 17325 17299 0 -1 4202496 2438664 1250 0 0 2026839 801 0 0 25 0 5 0 512344013 1639157760 374470 33554432000 4194304 4356188 140735304150512 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17330/statm: 400185 374470 164 40 0 398729 0
[pid=17330/tid=17338] ppid=17325 vsize=1600740 CPUtime=5049.54 cores=4-7
/proc/17330/task/17338/stat : 17338 (plingeling) R 17325 17325 17299 0 -1 4202560 829204 1250 0 0 504807 147 0 0 25 0 5 0 512344628 1639157760 374470 33554432000 4194304 4356188 140735304150512 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17330/tid=17339] ppid=17325 vsize=1600740 CPUtime=5071.37 cores=4-7
/proc/17330/task/17339/stat : 17339 (plingeling) R 17325 17325 17299 0 -1 4202560 429284 1250 0 0 506854 283 0 0 25 0 5 0 512344628 1639157760 374470 33554432000 4194304 4356188 140735304150512 18446744073709551615 4234310 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17330/tid=17340] ppid=17325 vsize=1600740 CPUtime=5073.38 cores=4-7
/proc/17330/task/17340/stat : 17340 (plingeling) R 17325 17325 17299 0 -1 4202560 425987 1250 0 0 507094 244 0 0 25 0 5 0 512344628 1639157760 374470 33554432000 4194304 4356188 140735304150512 18446744073709551615 4279551 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17330/tid=17341] ppid=17325 vsize=1600740 CPUtime=5075.92 cores=4-7
/proc/17330/task/17341/stat : 17341 (plingeling) R 17325 17325 17299 0 -1 4202560 552196 1250 0 0 507495 97 0 0 25 0 5 0 512344628 1639157760 374470 33554432000 4194304 4356188 140735304150512 18446744073709551615 4281706 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 30341.1
Current children cumulated vsize (KiB) 5634696



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 6.03 6.03 6.00 7/176 17489
/proc/meminfo: memFree=8808912/32951132 swapFree=67111528/67111528
[pid=17325] ppid=17323 vsize=18324 CPUtime=208.02 cores=0-7
/proc/17325/stat : 17325 (ppfolio) S 17323 17325 17299 0 -1 4202496 521 2527177 0 0 20 45 20189 548 15 0 1 0 512344012 18763776 390 33554432000 4194304 4287625 140735083597744 18446744073709551615 274664828431 0 0 4096 65536 4606796765096574976 0 0 17 0 0 0 0
/proc/17325/statm: 4581 390 342 23 0 91 0
[pid=17326] ppid=17325 vsize=1992020 CPUtime=5099.45 cores=0
/proc/17326/stat : 17326 (cryptominisat) R 17325 17325 17299 0 -1 4202496 284090839 0 0 0 473724 36221 0 0 25 0 1 0 512344013 2039828480 472006 33554432000 4194304 4587354 140733652513184 18446744073709551615 274664505736 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/17326/statm: 498005 472006 340 96 0 494567 0
[pid=17327] ppid=17325 vsize=2025556 CPUtime=4792.61 cores=1
/proc/17327/stat : 17327 (clasp) R 17325 17325 17299 0 -1 4202496 533358 0 0 0 474119 5142 0 0 25 0 1 0 512344013 2074169344 502493 33554432000 4194304 4785740 140733201668592 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/17327/statm: 506389 502493 336 145 0 503434 0
[pid=17330] ppid=17325 vsize=1600740 CPUtime=20347.1 cores=4-7
/proc/17330/stat : 17330 (plingeling) S 17325 17325 17299 0 -1 4202496 2442728 1250 0 0 2033913 801 0 0 25 0 5 0 512344013 1639157760 378534 33554432000 4194304 4356188 140735304150512 18446744073709551615 274676611045 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/17330/statm: 400185 378534 164 40 0 398729 0
[pid=17330/tid=17338] ppid=17325 vsize=1600740 CPUtime=5067.15 cores=4-7
/proc/17330/task/17338/stat : 17338 (plingeling) R 17325 17325 17299 0 -1 4202560 833268 1250 0 0 506567 148 0 0 25 0 5 0 512344628 1639157760 378534 33554432000 4194304 4356188 140735304150512 18446744073709551615 4210575 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=17330/tid=17339] ppid=17325 vsize=1600740 CPUtime=5089.08 cores=4-7
/proc/17330/task/17339/stat : 17339 (plingeling) R 17325 17325 17299 0 -1 4202560 429284 1250 0 0 508625 283 0 0 25 0 5 0 512344628 1639157760 378534 33554432000 4194304 4356188 140735304150512 18446744073709551615 4331106 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=17330/tid=17340] ppid=17325 vsize=1600740 CPUtime=5091.09 cores=4-7
/proc/17330/task/17340/stat : 17340 (plingeling) R 17325 17325 17299 0 -1 4202560 425987 1250 0 0 508865 244 0 0 25 0 5 0 512344628 1639157760 378534 33554432000 4194304 4356188 140735304150512 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=17330/tid=17341] ppid=17325 vsize=1600740 CPUtime=5093.64 cores=4-7
/proc/17330/task/17341/stat : 17341 (plingeling) R 17325 17325 17299 0 -1 4202560 552196 1250 0 0 509267 97 0 0 25 0 5 0 512344628 1639157760 378534 33554432000 4194304 4356188 140735304150512 18446744073709551615 4241569 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 30447.2
Current children cumulated vsize (KiB) 5636640

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 17325 and gives
#  childrusage.ru_utime.tv_sec=202
#  childrusage.ru_utime.tv_usec=105275
#  childrusage.ru_stime.tv_sec=5
#  childrusage.ru_stime.tv_usec=937097
# CPU time returned by wait4() is 208.042
# while last known CPU time is 30447.2
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5100.02
CPU time (s): 30447.2
CPU user time (s): 30019.7
CPU system time (s): 427.57
CPU usage (%): 597.001
Max. virtual memory (cumulated for all children) (KiB): 9963352

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 202.105
system time used= 5.9371
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2527698
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 127281
involuntary context switches= 521

runsolver used 7.59085 second user time and 19.1361 second system time

The end

Launcher Data 

Begin job on node131 at 2011-04-28 18:17:24
IDJOB=3310244
IDBENCH=82413
IDSOLVER=1638
FILE ID=node131/3310244-1304007440
RUNJOBID= node131-1304007440-17313
PBS_JOBID= 13168835
Free space on /tmp= 71268 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.030-NOTKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3310244-1304007440/watcher-3310244-1304007440 -o /tmp/evaluation-result-3310244-1304007440/solver-3310244-1304007440 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3310244-1304007440.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= c6d3480970bb2819448dd0a800a3cf1c
RANDOM SEED=1510388010

node131.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      15804208 kB
Buffers:       1026836 kB
Cached:       15529636 kB
SwapCached:          0 kB
Active:        6770640 kB
Inactive:      9851636 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      15804208 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:          130948 kB
Writeback:           0 kB
AnonPages:       65696 kB
Mapped:          14708 kB
Slab:           459688 kB
PageTables:       4316 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS:   185288 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 69720 MiB
End job on node131 at 2011-04-28 19:42:26