Trace number 3275166

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 40000.1 5003.72

General information on the benchmark

NameCRAFTED/rbsat/crafted/
forced/rbsat-v2100c215164gyes4.cnf
MD5SUM2caae162eff2e5f641cadfad345a5401
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2100
Number of clauses215164
Sum of the clauses size432288
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2215094
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 570

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3275166-1304145348/watcher-3275166-1304145348 -o /tmp/evaluation-result-3275166-1304145348/solver-3275166-1304145348 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3275166-1304145348.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.51 7.06 5.42 2/170 29007
/proc/meminfo: memFree=30724768/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) R 29005 29007 28885 0 -1 4194304 504 0 0 0 0 0 0 0 25 0 1 0 604765527 18763776 387 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284746133 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/29007/statm: 4581 387 339 23 0 91 0

[startup+0.061815 s]
/proc/loadavg: 7.51 7.06 5.42 2/170 29007
/proc/meminfo: memFree=30724768/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 25 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 6 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100792 s]
/proc/loadavg: 7.51 7.06 5.42 2/170 29007
/proc/meminfo: memFree=30724768/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 25 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300729 s]
/proc/loadavg: 7.51 7.06 5.42 2/170 29007
/proc/meminfo: memFree=30724768/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 24 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700658 s]
/proc/loadavg: 7.51 7.06 5.42 2/170 29007
/proc/meminfo: memFree=30724768/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 20 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50053 s]
/proc/loadavg: 7.51 7.06 5.42 9/180 29022
/proc/meminfo: memFree=30598456/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
[pid=29008] ppid=29007 vsize=58424 CPUtime=1.48 cores=0
/proc/29008/stat : 29008 (cryptominisat) R 29007 29007 28885 0 -1 4202496 12658 0 0 0 147 1 0 0 20 0 1 0 604765528 59826176 9679 33554432000 4194304 4587354 140734606127456 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29008/statm: 14606 9679 326 96 0 11168 0
[pid=29009] ppid=29007 vsize=27228 CPUtime=1.48 cores=1
/proc/29009/stat : 29009 (clasp) R 29007 29007 28885 0 -1 4202496 5745 0 0 0 148 0 0 0 20 0 1 0 604765528 27881472 4047 33554432000 4194304 4785740 140735232757040 18446744073709551615 4287050 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/29009/statm: 6807 4047 340 145 0 3852 0
[pid=29010] ppid=29007 vsize=463648 CPUtime=1.48 cores=2
/proc/29010/stat : 29010 (TNM) R 29007 29007 28885 0 -1 4202496 5218 0 0 0 148 0 0 0 20 0 1 0 604765528 474775552 5115 33554432000 134512640 134534564 4290158816 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/29010/statm: 115912 5115 95 6 0 114578 0
[pid=29011] ppid=29007 vsize=37004 CPUtime=1.48 cores=3
/proc/29011/stat : 29011 (march_hi) R 29007 29007 28885 0 -1 4202496 17503 0 0 0 146 2 0 0 25 0 1 0 604765528 37892096 4052 33554432000 4194304 4331988 140733657466144 18446744073709551615 4210448 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29011/statm: 9251 4052 174 34 0 7693 0
[pid=29012] ppid=29007 vsize=159124 CPUtime=5.48 cores=4-7
/proc/29012/stat : 29012 (plingeling) S 29007 29007 28885 0 -1 4202496 16175 1255 0 0 546 2 0 0 18 0 5 0 604765528 162942976 11822 33554432000 4194304 4356188 140737412159808 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29012/statm: 39781 11822 162 40 0 38325 0
[pid=29012/tid=29019] ppid=29007 vsize=159124 CPUtime=1.33 cores=4-7
/proc/29012/task/29019/stat : 29019 (plingeling) R 29007 29007 28885 0 -1 4202560 3333 1255 0 0 133 0 0 0 19 0 5 0 604765542 162942976 11822 33554432000 4194304 4356188 140737412159808 18446744073709551615 4270872 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29012/tid=29020] ppid=29007 vsize=159124 CPUtime=1.3 cores=4-7
/proc/29012/task/29020/stat : 29020 (plingeling) R 29007 29007 28885 0 -1 4202560 2484 1255 0 0 130 0 0 0 25 0 5 0 604765542 162942976 11822 33554432000 4194304 4356188 140737412159808 18446744073709551615 4210544 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29012/tid=29021] ppid=29007 vsize=159124 CPUtime=1.33 cores=4-7
/proc/29012/task/29021/stat : 29021 (plingeling) R 29007 29007 28885 0 -1 4202560 3271 1255 0 0 133 0 0 0 23 0 5 0 604765542 162942976 11822 33554432000 4194304 4356188 140737412159808 18446744073709551615 4276747 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29012/tid=29022] ppid=29007 vsize=159124 CPUtime=1.34 cores=4-7
/proc/29012/task/29022/stat : 29022 (plingeling) R 29007 29007 28885 0 -1 4202560 2393 1255 0 0 134 0 0 0 22 0 5 0 604765542 162942976 11822 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331319 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.4
Current children cumulated vsize (KiB) 763752

[startup+3.10126 s]
/proc/loadavg: 7.51 7.06 5.42 9/180 29022
/proc/meminfo: memFree=30586332/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
[pid=29008] ppid=29007 vsize=1246376 CPUtime=3.09 cores=0
/proc/29008/stat : 29008 (cryptominisat) R 29007 29007 28885 0 -1 4202496 14554 0 0 0 307 2 0 0 25 0 1 0 604765528 1276289024 9896 33554432000 4194304 4587354 140734606127456 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29008/statm: 311594 9896 337 96 0 308156 0
[pid=29009] ppid=29007 vsize=28852 CPUtime=3.08 cores=1
/proc/29009/stat : 29009 (clasp) R 29007 29007 28885 0 -1 4202496 6245 0 0 0 308 0 0 0 25 0 1 0 604765528 29544448 4547 33554432000 4194304 4785740 140735232757040 18446744073709551615 4309533 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/29009/statm: 7213 4547 340 145 0 4258 0
[pid=29010] ppid=29007 vsize=467480 CPUtime=3.08 cores=2
/proc/29010/stat : 29010 (TNM) R 29007 29007 28885 0 -1 4202496 5219 0 0 0 308 0 0 0 25 0 1 0 604765528 478699520 5116 33554432000 134512640 134534564 4290158816 18446744073709551615 134528979 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/29010/statm: 116870 5116 95 6 0 115536 0
[pid=29011] ppid=29007 vsize=37308 CPUtime=3.08 cores=3
/proc/29011/stat : 29011 (march_hi) R 29007 29007 28885 0 -1 4202496 17571 0 0 0 306 2 0 0 25 0 1 0 604765528 38203392 4120 33554432000 4194304 4331988 140733657466144 18446744073709551615 4226479 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29011/statm: 9327 4120 174 34 0 7769 0
[pid=29012] ppid=29007 vsize=175508 CPUtime=11.88 cores=4-7
/proc/29012/stat : 29012 (plingeling) S 29007 29007 28885 0 -1 4202496 20271 1255 0 0 1185 3 0 0 18 0 5 0 604765528 179720192 15918 33554432000 4194304 4356188 140737412159808 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29012/statm: 43877 15918 162 40 0 42421 0
[pid=29012/tid=29019] ppid=29007 vsize=175508 CPUtime=2.94 cores=4-7
/proc/29012/task/29019/stat : 29019 (plingeling) R 29007 29007 28885 0 -1 4202560 3588 1255 0 0 293 1 0 0 24 0 5 0 604765542 179720192 15918 33554432000 4194304 4356188 140737412159808 18446744073709551615 4323559 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29012/tid=29020] ppid=29007 vsize=175508 CPUtime=2.9 cores=4-7
/proc/29012/task/29020/stat : 29020 (plingeling) R 29007 29007 28885 0 -1 4202560 3124 1255 0 0 290 0 0 0 25 0 5 0 604765542 179720192 15918 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29012/tid=29021] ppid=29007 vsize=175508 CPUtime=2.93 cores=4-7
/proc/29012/task/29021/stat : 29021 (plingeling) R 29007 29007 28885 0 -1 4202560 3783 1255 0 0 293 0 0 0 25 0 5 0 604765542 179720192 15918 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331992 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29012/tid=29022] ppid=29007 vsize=175508 CPUtime=2.94 cores=4-7
/proc/29012/task/29022/stat : 29022 (plingeling) R 29007 29007 28885 0 -1 4202560 5082 1255 0 0 294 0 0 0 25 0 5 0 604765542 179720192 15918 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331264 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.21
Current children cumulated vsize (KiB) 1973848

[startup+6.30073 s]
/proc/loadavg: 7.71 7.11 5.44 9/180 29022
/proc/meminfo: memFree=30547408/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
[pid=29008] ppid=29007 vsize=1257052 CPUtime=6.29 cores=0
/proc/29008/stat : 29008 (cryptominisat) R 29007 29007 28885 0 -1 4202496 16799 0 0 0 627 2 0 0 25 0 1 0 604765528 1287221248 12137 33554432000 4194304 4587354 140734606127456 18446744073709551615 4365464 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29008/statm: 314263 12137 337 96 0 310825 0
[pid=29009] ppid=29007 vsize=41068 CPUtime=6.29 cores=1
/proc/29009/stat : 29009 (clasp) R 29007 29007 28885 0 -1 4202496 9298 0 0 0 628 1 0 0 25 0 1 0 604765528 42053632 7600 33554432000 4194304 4785740 140735232757040 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/29009/statm: 10267 7600 340 145 0 7312 0
[pid=29010] ppid=29007 vsize=467480 CPUtime=6.28 cores=2
/proc/29010/stat : 29010 (TNM) R 29007 29007 28885 0 -1 4202496 5219 0 0 0 628 0 0 0 25 0 1 0 604765528 478699520 5116 33554432000 134512640 134534564 4290158816 18446744073709551615 134528737 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/29010/statm: 116870 5116 95 6 0 115536 0
[pid=29011] ppid=29007 vsize=37712 CPUtime=6.28 cores=3
/proc/29011/stat : 29011 (march_hi) R 29007 29007 28885 0 -1 4202496 17667 0 0 0 626 2 0 0 25 0 1 0 604765528 38617088 4215 33554432000 4194304 4331988 140733657466144 18446744073709551615 4212625 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29011/statm: 9428 4215 174 34 0 7870 0
[pid=29012] ppid=29007 vsize=181140 CPUtime=24.66 cores=4-7
/proc/29012/stat : 29012 (plingeling) S 29007 29007 28885 0 -1 4202496 21666 1255 0 0 2463 3 0 0 18 0 5 0 604765528 185487360 17313 33554432000 4194304 4356188 140737412159808 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29012/statm: 45285 17313 162 40 0 43829 0
[pid=29012/tid=29019] ppid=29007 vsize=181140 CPUtime=6.14 cores=4-7
/proc/29012/task/29019/stat : 29019 (plingeling) R 29007 29007 28885 0 -1 4202560 4229 1255 0 0 613 1 0 0 25 0 5 0 604765542 185487360 17313 33554432000 4194304 4356188 140737412159808 18446744073709551615 4210587 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29012/tid=29020] ppid=29007 vsize=181140 CPUtime=6.08 cores=4-7
/proc/29012/task/29020/stat : 29020 (plingeling) R 29007 29007 28885 0 -1 4202560 3636 1255 0 0 608 0 0 0 25 0 5 0 604765542 185487360 17313 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331002 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29012/tid=29021] ppid=29007 vsize=181140 CPUtime=6.13 cores=4-7
/proc/29012/task/29021/stat : 29021 (plingeling) R 29007 29007 28885 0 -1 4202560 4025 1255 0 0 613 0 0 0 25 0 5 0 604765542 185487360 17313 33554432000 4194304 4356188 140737412159808 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29012/tid=29022] ppid=29007 vsize=181140 CPUtime=6.14 cores=4-7
/proc/29012/task/29022/stat : 29022 (plingeling) R 29007 29007 28885 0 -1 4202560 5082 1255 0 0 614 0 0 0 25 0 5 0 604765542 185487360 17313 33554432000 4194304 4356188 140737412159808 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 49.8
Current children cumulated vsize (KiB) 2002776

[startup+12.7007 s]
/proc/loadavg: 7.73 7.12 5.45 9/180 29022
/proc/meminfo: memFree=30502580/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
[pid=29008] ppid=29007 vsize=1277908 CPUtime=12.68 cores=0
/proc/29008/stat : 29008 (cryptominisat) R 29007 29007 28885 0 -1 4202496 23147 0 0 0 1265 3 0 0 25 0 1 0 604765528 1308577792 16066 33554432000 4194304 4587354 140734606127456 18446744073709551615 4365216 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29008/statm: 319477 16066 337 96 0 316039 0
[pid=29009] ppid=29007 vsize=57452 CPUtime=12.69 cores=1
/proc/29009/stat : 29009 (clasp) R 29007 29007 28885 0 -1 4202496 13356 0 0 0 1266 3 0 0 25 0 1 0 604765528 58830848 11658 33554432000 4194304 4785740 140735232757040 18446744073709551615 4286919 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/29009/statm: 14363 11658 340 145 0 11408 0
[pid=29010] ppid=29007 vsize=467480 CPUtime=12.68 cores=2
/proc/29010/stat : 29010 (TNM) R 29007 29007 28885 0 -1 4202496 5219 0 0 0 1268 0 0 0 25 0 1 0 604765528 478699520 5116 33554432000 134512640 134534564 4290158816 18446744073709551615 134528708 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/29010/statm: 116870 5116 95 6 0 115536 0
[pid=29011] ppid=29007 vsize=38556 CPUtime=12.69 cores=3
/proc/29011/stat : 29011 (march_hi) R 29007 29007 28885 0 -1 4202496 17873 0 0 0 1266 3 0 0 25 0 1 0 604765528 39481344 4420 33554432000 4194304 4331988 140733657466144 18446744073709551615 4265367 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29011/statm: 9639 4420 174 34 0 8081 0
[pid=29012] ppid=29007 vsize=195644 CPUtime=50.22 cores=4-7
/proc/29012/stat : 29012 (plingeling) S 29007 29007 28885 0 -1 4202496 24795 1255 0 0 5018 4 0 0 18 0 5 0 604765528 200339456 20442 33554432000 4194304 4356188 140737412159808 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29012/statm: 48911 20442 163 40 0 47455 0
[pid=29012/tid=29019] ppid=29007 vsize=195644 CPUtime=12.54 cores=4-7
/proc/29012/task/29019/stat : 29019 (plingeling) R 29007 29007 28885 0 -1 4202560 4229 1255 0 0 1253 1 0 0 25 0 5 0 604765542 200339456 20442 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331076 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29012/tid=29020] ppid=29007 vsize=195644 CPUtime=12.44 cores=4-7
/proc/29012/task/29020/stat : 29020 (plingeling) R 29007 29007 28885 0 -1 4202560 4404 1255 0 0 1244 0 0 0 25 0 5 0 604765542 200339456 20442 33554432000 4194304 4356188 140737412159808 18446744073709551615 4244620 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29012/tid=29021] ppid=29007 vsize=195644 CPUtime=12.53 cores=4-7
/proc/29012/task/29021/stat : 29021 (plingeling) R 29007 29007 28885 0 -1 4202560 4281 1255 0 0 1253 0 0 0 25 0 5 0 604765542 200339456 20442 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29012/tid=29022] ppid=29007 vsize=195644 CPUtime=12.53 cores=4-7
/proc/29012/task/29022/stat : 29022 (plingeling) R 29007 29007 28885 0 -1 4202560 7187 1255 0 0 1253 0 0 0 25 0 5 0 604765542 200339456 20442 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 100.96
Current children cumulated vsize (KiB) 2055364

[startup+25.5005 s]
/proc/loadavg: 7.79 7.16 5.49 9/180 29022
/proc/meminfo: memFree=30454924/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
[pid=29008] ppid=29007 vsize=1287692 CPUtime=25.49 cores=0
/proc/29008/stat : 29008 (cryptominisat) R 29007 29007 28885 0 -1 4202496 31093 0 0 0 2544 5 0 0 25 0 1 0 604765528 1318596608 21547 33554432000 4194304 4587354 140734606127456 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=29010] ppid=29007 vsize=467480 CPUtime=4842.35 cores=2
/proc/29010/stat : 29010 (TNM) R 29007 29007 28885 0 -1 4202496 5219 0 0 0 484233 2 0 0 25 0 1 0 604765528 478699520 5116 33554432000 134512640 134534564 4290158816 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/29010/statm: 116870 5116 95 6 0 115536 0
[pid=29011] ppid=29007 vsize=435096 CPUtime=4842.21 cores=3
/proc/29011/stat : 29011 (march_hi) R 29007 29007 28885 0 -1 4202496 85512 0 0 0 484169 52 0 0 25 0 1 0 604765528 445538304 72059 33554432000 4194304 4331988 140733657466144 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29011/statm: 108774 72059 175 34 0 107216 0
[pid=29012] ppid=29007 vsize=574460 CPUtime=19340.5 cores=4-7
/proc/29012/stat : 29012 (plingeling) S 29007 29007 28885 0 -1 4202496 3366348 1255 0 0 1933594 452 0 0 18 0 5 0 604765528 588247040 115689 33554432000 4194304 4356188 140737412159808 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29012/statm: 143615 115689 163 40 0 142159 0
[pid=29012/tid=29019] ppid=29007 vsize=574460 CPUtime=4842.2 cores=4-7
/proc/29012/task/29019/stat : 29019 (plingeling) R 29007 29007 28885 0 -1 4202560 862430 1255 0 0 484114 106 0 0 25 0 5 0 604765542 588247040 115689 33554432000 4194304 4356188 140737412159808 18446744073709551615 4273767 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29012/tid=29020] ppid=29007 vsize=574460 CPUtime=4814.15 cores=4-7
/proc/29012/task/29020/stat : 29020 (plingeling) R 29007 29007 28885 0 -1 4202560 805823 1255 0 0 481310 105 0 0 25 0 5 0 604765542 588247040 115689 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29012/tid=29021] ppid=29007 vsize=574460 CPUtime=4842.2 cores=4-7
/proc/29012/task/29021/stat : 29021 (plingeling) R 29007 29007 28885 0 -1 4202560 904895 1255 0 0 484096 124 0 0 25 0 5 0 604765542 588247040 115689 33554432000 4194304 4356188 140737412159808 18446744073709551615 4332288 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29012/tid=29022] ppid=29007 vsize=574460 CPUtime=4841.74 cores=4-7
/proc/29012/task/29022/stat : 29022 (plingeling) R 29007 29007 28885 0 -1 4202560 788506 1255 0 0 484060 114 0 0 25 0 5 0 604765542 588247040 115689 33554432000 4194304 4356188 140737412159808 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38709.7
Current children cumulated vsize (KiB) 3360908

[startup+4902.3 s]
/proc/loadavg: 8.11 8.06 8.00 9/180 29167
/proc/meminfo: memFree=29252472/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
[pid=29008] ppid=29007 vsize=1551272 CPUtime=4902.35 cores=0
/proc/29008/stat : 29008 (cryptominisat) R 29007 29007 28885 0 -1 4202496 1373095 0 0 0 489602 633 0 0 25 0 1 0 604765528 1588502528 91042 33554432000 4194304 4587354 140734606127456 18446744073709551615 4243360 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29008/statm: 387818 91042 337 96 0 384380 0
[pid=29009] ppid=29007 vsize=316300 CPUtime=4902.35 cores=1
/proc/29009/stat : 29009 (clasp) R 29007 29007 28885 0 -1 4202496 74274 0 0 0 490209 26 0 0 25 0 1 0 604765528 323891200 72576 33554432000 4194304 4785740 140735232757040 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/29009/statm: 79075 72576 340 145 0 76120 0
[pid=29010] ppid=29007 vsize=467480 CPUtime=4902.35 cores=2
/proc/29010/stat : 29010 (TNM) R 29007 29007 28885 0 -1 4202496 5219 0 0 0 490233 2 0 0 25 0 1 0 604765528 478699520 5116 33554432000 134512640 134534564 4290158816 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/29010/statm: 116870 5116 95 6 0 115536 0
[pid=29011] ppid=29007 vsize=435096 CPUtime=4902.21 cores=3
/proc/29011/stat : 29011 (march_hi) R 29007 29007 28885 0 -1 4202496 86422 0 0 0 490169 52 0 0 25 0 1 0 604765528 445538304 72969 33554432000 4194304 4331988 140733657466144 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29011/statm: 108774 72969 175 34 0 107216 0
[pid=29012] ppid=29007 vsize=627708 CPUtime=19580.2 cores=4-7
/proc/29012/stat : 29012 (plingeling) S 29007 29007 28885 0 -1 4202496 3405772 1255 0 0 1957558 457 0 0 18 0 5 0 604765528 642772992 129001 33554432000 4194304 4356188 140737412159808 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29012/statm: 156927 129001 163 40 0 155471 0
[pid=29012/tid=29019] ppid=29007 vsize=627708 CPUtime=4902.21 cores=4-7
/proc/29012/task/29019/stat : 29019 (plingeling) R 29007 29007 28885 0 -1 4202560 872158 1255 0 0 490114 107 0 0 25 0 5 0 604765542 642772992 129001 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29012/tid=29020] ppid=29007 vsize=627708 CPUtime=4873.82 cores=4-7
/proc/29012/task/29020/stat : 29020 (plingeling) R 29007 29007 28885 0 -1 4202560 814015 1255 0 0 487276 106 0 0 25 0 5 0 604765542 642772992 129001 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29012/tid=29021] ppid=29007 vsize=627708 CPUtime=4902.2 cores=4-7
/proc/29012/task/29021/stat : 29021 (plingeling) R 29007 29007 28885 0 -1 4202560 917183 1255 0 0 490094 126 0 0 25 0 5 0 604765542 642772992 129001 33554432000 4194304 4356188 140737412159808 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29012/tid=29022] ppid=29007 vsize=627708 CPUtime=4901.74 cores=4-7
/proc/29012/task/29022/stat : 29022 (plingeling) R 29007 29007 28885 0 -1 4202560 797722 1255 0 0 490059 115 0 0 25 0 5 0 604765542 642772992 129001 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39189.4
Current children cumulated vsize (KiB) 3416180

[startup+4962.3 s]
/proc/loadavg: 8.13 8.08 8.01 9/180 29169
/proc/meminfo: memFree=29273264/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
[pid=29008] ppid=29007 vsize=1549244 CPUtime=4962.35 cores=0
/proc/29008/stat : 29008 (cryptominisat) R 29007 29007 28885 0 -1 4202496 1389189 0 0 0 495594 641 0 0 25 0 1 0 604765528 1586425856 96133 33554432000 4194304 4587354 140734606127456 18446744073709551615 4238606 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29008/statm: 387311 96133 337 96 0 383873 0
[pid=29009] ppid=29007 vsize=316300 CPUtime=4962.35 cores=1
/proc/29009/stat : 29009 (clasp) R 29007 29007 28885 0 -1 4202496 74283 0 0 0 496209 26 0 0 25 0 1 0 604765528 323891200 72585 33554432000 4194304 4785740 140735232757040 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/29009/statm: 79075 72585 340 145 0 76120 0
[pid=29010] ppid=29007 vsize=467480 CPUtime=4962.35 cores=2
/proc/29010/stat : 29010 (TNM) R 29007 29007 28885 0 -1 4202496 5219 0 0 0 496233 2 0 0 25 0 1 0 604765528 478699520 5116 33554432000 134512640 134534564 4290158816 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/29010/statm: 116870 5116 95 6 0 115536 0
[pid=29011] ppid=29007 vsize=435096 CPUtime=4962.21 cores=3
/proc/29011/stat : 29011 (march_hi) R 29007 29007 28885 0 -1 4202496 87590 0 0 0 496168 53 0 0 25 0 1 0 604765528 445538304 74137 33554432000 4194304 4331988 140733657466144 18446744073709551615 4246743 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29011/statm: 108774 74137 175 34 0 107216 0
[pid=29012] ppid=29007 vsize=580604 CPUtime=19819.8 cores=4-7
/proc/29012/stat : 29012 (plingeling) S 29007 29007 28885 0 -1 4202496 3445196 1255 0 0 1981519 462 0 0 18 0 5 0 604765528 594538496 117225 33554432000 4194304 4356188 140737412159808 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29012/statm: 145151 117225 163 40 0 143695 0
[pid=29012/tid=29019] ppid=29007 vsize=580604 CPUtime=4962.21 cores=4-7
/proc/29012/task/29019/stat : 29019 (plingeling) R 29007 29007 28885 0 -1 4202560 881886 1255 0 0 496112 109 0 0 25 0 5 0 604765542 594538496 117225 33554432000 4194304 4356188 140737412159808 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29012/tid=29020] ppid=29007 vsize=580604 CPUtime=4933.5 cores=4-7
/proc/29012/task/29020/stat : 29020 (plingeling) R 29007 29007 28885 0 -1 4202560 830399 1255 0 0 493242 108 0 0 25 0 5 0 604765542 594538496 117225 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29012/tid=29021] ppid=29007 vsize=580604 CPUtime=4962.2 cores=4-7
/proc/29012/task/29021/stat : 29021 (plingeling) R 29007 29007 28885 0 -1 4202560 925375 1255 0 0 496093 127 0 0 25 0 5 0 604765542 594538496 117225 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29012/tid=29022] ppid=29007 vsize=580604 CPUtime=4961.73 cores=4-7
/proc/29012/task/29022/stat : 29022 (plingeling) R 29007 29007 28885 0 -1 4202560 802842 1255 0 0 496057 116 0 0 25 0 5 0 604765542 594538496 117225 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39669.1
Current children cumulated vsize (KiB) 3367048



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.7 s]
/proc/loadavg: 8.20 8.10 8.02 9/180 29169
/proc/meminfo: memFree=29297896/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
[pid=29008] ppid=29007 vsize=1554080 CPUtime=5003.75 cores=0
/proc/29008/stat : 29008 (cryptominisat) R 29007 29007 28885 0 -1 4202496 1399147 0 0 0 499729 646 0 0 25 0 1 0 604765528 1591377920 95293 33554432000 4194304 4587354 140734606127456 18446744073709551615 4238421 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29008/statm: 388520 95293 337 96 0 385082 0
[pid=29009] ppid=29007 vsize=318432 CPUtime=5003.76 cores=1
/proc/29009/stat : 29009 (clasp) R 29007 29007 28885 0 -1 4202496 74710 0 0 0 500350 26 0 0 25 0 1 0 604765528 326074368 73012 33554432000 4194304 4785740 140735232757040 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/29009/statm: 79608 73012 340 145 0 76653 0
[pid=29010] ppid=29007 vsize=467480 CPUtime=5003.76 cores=2
/proc/29010/stat : 29010 (TNM) R 29007 29007 28885 0 -1 4202496 5219 0 0 0 500374 2 0 0 25 0 1 0 604765528 478699520 5116 33554432000 134512640 134534564 4290158816 18446744073709551615 134528302 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/29010/statm: 116870 5116 95 6 0 115536 0
[pid=29011] ppid=29007 vsize=435096 CPUtime=5003.61 cores=3
/proc/29011/stat : 29011 (march_hi) R 29007 29007 28885 0 -1 4202496 88580 0 0 0 500307 54 0 0 25 0 1 0 604765528 445538304 75127 33554432000 4194304 4331988 140733657466144 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29011/statm: 108774 75127 175 34 0 107216 0
[pid=29012] ppid=29007 vsize=543744 CPUtime=19985.2 cores=4-7
/proc/29012/stat : 29012 (plingeling) S 29007 29007 28885 0 -1 4202496 3479501 1255 0 0 1998055 465 0 0 18 0 5 0 604765528 556793856 108010 33554432000 4194304 4356188 140737412159808 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29012/statm: 135936 108010 163 40 0 134480 0
[pid=29012/tid=29019] ppid=29007 vsize=543744 CPUtime=5003.61 cores=4-7
/proc/29012/task/29019/stat : 29019 (plingeling) R 29007 29007 28885 0 -1 4202560 895710 1255 0 0 500251 110 0 0 25 0 5 0 604765542 556793856 108010 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331323 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29012/tid=29020] ppid=29007 vsize=543744 CPUtime=4974.67 cores=4-7
/proc/29012/task/29020/stat : 29020 (plingeling) R 29007 29007 28885 0 -1 4202560 830399 1255 0 0 497359 108 0 0 25 0 5 0 604765542 556793856 108010 33554432000 4194304 4356188 140737412159808 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29012/tid=29021] ppid=29007 vsize=543744 CPUtime=5003.61 cores=4-7
/proc/29012/task/29021/stat : 29021 (plingeling) R 29007 29007 28885 0 -1 4202560 937664 1255 0 0 500233 128 0 0 25 0 5 0 604765542 556793856 108010 33554432000 4194304 4356188 140737412159808 18446744073709551615 4281706 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29012/tid=29022] ppid=29007 vsize=543744 CPUtime=5003.13 cores=4-7
/proc/29012/task/29022/stat : 29022 (plingeling) R 29007 29007 28885 0 -1 4202560 811034 1255 0 0 500196 117 0 0 25 0 5 0 604765542 556793856 108010 33554432000 4194304 4356188 140737412159808 18446744073709551615 4233969 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 3337156

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5003.7 s]
/proc/loadavg: 8.20 8.10 8.02 9/180 29169
/proc/meminfo: memFree=29297896/32951124 swapFree=67103364/67111528
[pid=29007] ppid=29005 vsize=18324 CPUtime=0 cores=0-7
/proc/29007/stat : 29007 (ppfolio) S 29005 29007 28885 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 604765527 18763776 389 33554432000 4194304 4287625 140735685763968 18446744073709551615 224284945935 0 0 4096 65536 2164332018723192831 0 0 17 4 0 0 0
/proc/29007/statm: 4581 389 341 23 0 91 0
[pid=29008] ppid=29007 vsize=1554080 CPUtime=5003.75 cores=0
/proc/29008/stat : 29008 (cryptominisat) R 29007 29007 28885 0 -1 4202496 1399147 0 0 0 499729 646 0 0 25 0 1 0 604765528 1591377920 95293 33554432000 4194304 4587354 140734606127456 18446744073709551615 4238421 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29008/statm: 388520 95293 337 96 0 385082 0
[pid=29009] ppid=29007 vsize=318432 CPUtime=5003.76 cores=1
/proc/29009/stat : 29009 (clasp) R 29007 29007 28885 0 -1 4202496 74710 0 0 0 500350 26 0 0 25 0 1 0 604765528 326074368 73012 33554432000 4194304 4785740 140735232757040 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/29009/statm: 79608 73012 340 145 0 76653 0
[pid=29010] ppid=29007 vsize=467480 CPUtime=5003.76 cores=2
/proc/29010/stat : 29010 (TNM) R 29007 29007 28885 0 -1 4202496 5219 0 0 0 500374 2 0 0 25 0 1 0 604765528 478699520 5116 33554432000 134512640 134534564 4290158816 18446744073709551615 134528302 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/29010/statm: 116870 5116 95 6 0 115536 0
[pid=29011] ppid=29007 vsize=435096 CPUtime=5003.61 cores=3
/proc/29011/stat : 29011 (march_hi) R 29007 29007 28885 0 -1 4202496 88580 0 0 0 500307 54 0 0 25 0 1 0 604765528 445538304 75127 33554432000 4194304 4331988 140733657466144 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/29011/statm: 108774 75127 175 34 0 107216 0
[pid=29012] ppid=29007 vsize=543744 CPUtime=19985.2 cores=4-7
/proc/29012/stat : 29012 (plingeling) S 29007 29007 28885 0 -1 4202496 3479501 1255 0 0 1998055 465 0 0 18 0 5 0 604765528 556793856 108010 33554432000 4194304 4356188 140737412159808 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/29012/statm: 135936 108010 163 40 0 134480 0
[pid=29012/tid=29019] ppid=29007 vsize=543744 CPUtime=5003.61 cores=4-7
/proc/29012/task/29019/stat : 29019 (plingeling) R 29007 29007 28885 0 -1 4202560 895710 1255 0 0 500251 110 0 0 25 0 5 0 604765542 556793856 108010 33554432000 4194304 4356188 140737412159808 18446744073709551615 4331323 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=29012/tid=29020] ppid=29007 vsize=543744 CPUtime=4974.67 cores=4-7
/proc/29012/task/29020/stat : 29020 (plingeling) R 29007 29007 28885 0 -1 4202560 830399 1255 0 0 497359 108 0 0 25 0 5 0 604765542 556793856 108010 33554432000 4194304 4356188 140737412159808 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=29012/tid=29021] ppid=29007 vsize=543744 CPUtime=5003.61 cores=4-7
/proc/29012/task/29021/stat : 29021 (plingeling) R 29007 29007 28885 0 -1 4202560 937664 1255 0 0 500233 128 0 0 25 0 5 0 604765542 556793856 108010 33554432000 4194304 4356188 140737412159808 18446744073709551615 4281706 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29012/tid=29022] ppid=29007 vsize=543744 CPUtime=5003.13 cores=4-7
/proc/29012/task/29022/stat : 29022 (plingeling) R 29007 29007 28885 0 -1 4202560 811034 1255 0 0 500196 117 0 0 25 0 5 0 604765542 556793856 108010 33554432000 4194304 4356188 140737412159808 18446744073709551615 4233969 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 3337156

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 29007 and gives
#  childrusage.ru_utime.tv_sec=5003
#  childrusage.ru_utime.tv_usec=745315
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=36994
# CPU time returned by wait4() is 5003.78
# while last known CPU time is 40000.1
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5003.72
CPU time (s): 40000.1
CPU user time (s): 39988.1
CPU system time (s): 11.93
CPU usage (%): 799.407
Max. virtual memory (cumulated for all children) (KiB): 4561260

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5003.75
system time used= 0.036994
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 5740
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 520
involuntary context switches= 9279

runsolver used 8.12476 second user time and 19.1791 second system time

The end

Launcher Data 

Begin job on node118 at 2011-04-30 08:35:49
IDJOB=3275166
IDBENCH=71571
IDSOLVER=1638
FILE ID=node118/3275166-1304145348
RUNJOBID= node118-1304144825-28899
PBS_JOBID= 13170865
Free space on /tmp= 71932 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v2100c215164gyes4.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3275166-1304145348/watcher-3275166-1304145348 -o /tmp/evaluation-result-3275166-1304145348/solver-3275166-1304145348 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3275166-1304145348.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 2caae162eff2e5f641cadfad345a5401
RANDOM SEED=1416598289

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30724800 kB
Buffers:        422184 kB
Cached:        1389500 kB
SwapCached:       2568 kB
Active:         168444 kB
Inactive:      1702180 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30724800 kB
SwapTotal:    67111528 kB
SwapFree:     67103364 kB
Dirty:           12804 kB
Writeback:          56 kB
AnonPages:       57956 kB
Mapped:          14592 kB
Slab:           290128 kB
PageTables:       4460 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   187252 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71924 MiB
End job on node118 at 2011-04-30 09:59:14