Trace number 3276031

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 40000.3 5003.52

General information on the benchmark

NameSAT04/handmade/connamacher/connm-d0.04/
connm-ue-csp-sat-n800-d0.04-s1788334438.used-as.sat04-976.cnf
MD5SUM41d80cc808c021a52e21ee9950bc91e0
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark4759.42
Satisfiable
(Un)Satisfiability was proved
Number of variables764
Number of clauses9695
Sum of the clauses size29276
Maximum clause length4
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 39504
Number of clauses of size 4191
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3276031-1304172942/watcher-3276031-1304172942 -o /tmp/evaluation-result-3276031-1304172942/solver-3276031-1304172942 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3276031-1304172942.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.25 8.22 8.14 2/174 10813
/proc/meminfo: memFree=30812176/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) R 10811 10813 10103 0 -1 4202496 520 0 0 0 0 0 0 0 24 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0

[startup+0.052168 s]
/proc/loadavg: 8.25 8.22 8.14 2/174 10813
/proc/meminfo: memFree=30812176/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 24 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 7 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100161 s]
/proc/loadavg: 8.25 8.22 8.14 2/174 10813
/proc/meminfo: memFree=30812176/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 24 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 7 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300123 s]
/proc/loadavg: 8.25 8.22 8.14 2/174 10813
/proc/meminfo: memFree=30812176/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 23 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 7 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.701049 s]
/proc/loadavg: 8.25 8.22 8.14 2/174 10813
/proc/meminfo: memFree=30812176/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 20 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 7 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.5009 s]
/proc/loadavg: 8.25 8.22 8.14 10/184 10828
/proc/meminfo: memFree=30777120/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 7 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
[pid=10814] ppid=10813 vsize=1891848 CPUtime=1.49 cores=0
/proc/10814/stat : 10814 (cryptominisat) R 10813 10813 10103 0 -1 4202496 2552 0 0 0 148 1 0 0 20 0 1 0 528892073 1937252352 1162 33554432000 4194304 4587354 140735243217040 18446744073709551615 4236438 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10814/statm: 472962 1162 331 96 0 469524 0
[pid=10815] ppid=10813 vsize=14460 CPUtime=1.49 cores=1
/proc/10815/stat : 10815 (clasp) R 10813 10813 10103 0 -1 4202496 1087 0 0 0 149 0 0 0 20 0 1 0 528892073 14807040 973 33554432000 4194304 4785740 140736260543024 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10815/statm: 3615 973 340 145 0 660 0
[pid=10816] ppid=10813 vsize=456356 CPUtime=1.49 cores=2
/proc/10816/stat : 10816 (TNM) R 10813 10813 10103 0 -1 4202496 549 0 0 0 149 0 0 0 20 0 1 0 528892073 467308544 446 33554432000 134512640 134534564 4287614208 18446744073709551615 134528955 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10816/statm: 114089 446 95 6 0 112755 0
[pid=10817] ppid=10813 vsize=21848 CPUtime=1.49 cores=3
/proc/10817/stat : 10817 (march_hi) R 10813 10813 10103 0 -1 4202496 6081 0 0 0 148 1 0 0 20 0 1 0 528892073 22372352 3899 33554432000 4194304 4331988 140735581680816 18446744073709551615 4226231 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10817/statm: 5462 3899 174 34 0 3904 0
[pid=10818] ppid=10813 vsize=126740 CPUtime=5.9 cores=4-7
/proc/10818/stat : 10818 (plingeling) S 10813 10813 10103 0 -1 4202496 4768 1249 0 0 590 0 0 0 18 0 5 0 528892073 129781760 4125 33554432000 4194304 4356188 140737295992784 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/10818/statm: 31685 4125 164 40 0 30229 0
[pid=10818/tid=10825] ppid=10813 vsize=126740 CPUtime=1.48 cores=4-7
/proc/10818/task/10825/stat : 10825 (plingeling) R 10813 10813 10103 0 -1 4202560 979 1249 0 0 148 0 0 0 23 0 5 0 528892074 129781760 4125 33554432000 4194304 4356188 140737295992784 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10818/tid=10826] ppid=10813 vsize=126740 CPUtime=1.47 cores=4-7
/proc/10818/task/10826/stat : 10826 (plingeling) R 10813 10813 10103 0 -1 4202560 1028 1249 0 0 147 0 0 0 23 0 5 0 528892074 129781760 4125 33554432000 4194304 4356188 140737295992784 18446744073709551615 4279615 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10818/tid=10827] ppid=10813 vsize=126740 CPUtime=1.47 cores=4-7
/proc/10818/task/10827/stat : 10827 (plingeling) R 10813 10813 10103 0 -1 4202560 632 1249 0 0 147 0 0 0 23 0 5 0 528892074 129781760 4125 33554432000 4194304 4356188 140737295992784 18446744073709551615 4210549 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10818/tid=10828] ppid=10813 vsize=126740 CPUtime=1.46 cores=4-7
/proc/10818/task/10828/stat : 10828 (plingeling) R 10813 10813 10103 0 -1 4202560 1143 1249 0 0 146 0 0 0 23 0 5 0 528892074 129781760 4125 33554432000 4194304 4356188 140737295992784 18446744073709551615 4279656 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.86
Current children cumulated vsize (KiB) 2529576

[startup+3.10061 s]
/proc/loadavg: 8.23 8.21 8.14 9/184 10828
/proc/meminfo: memFree=30770052/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 7 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
[pid=10814] ppid=10813 vsize=1893712 CPUtime=3.09 cores=0
/proc/10814/stat : 10814 (cryptominisat) R 10813 10813 10103 0 -1 4202496 3807 0 0 0 308 1 0 0 25 0 1 0 528892073 1939161088 1554 33554432000 4194304 4587354 140735243217040 18446744073709551615 4230260 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10814/statm: 473428 1555 338 96 0 469990 0
[pid=10815] ppid=10813 vsize=15004 CPUtime=3.09 cores=1
/proc/10815/stat : 10815 (clasp) R 10813 10813 10103 0 -1 4202496 1215 0 0 0 309 0 0 0 25 0 1 0 528892073 15364096 1101 33554432000 4194304 4785740 140736260543024 18446744073709551615 4553366 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10815/statm: 3751 1101 340 145 0 796 0
[pid=10816] ppid=10813 vsize=456356 CPUtime=3.09 cores=2
/proc/10816/stat : 10816 (TNM) R 10813 10813 10103 0 -1 4202496 549 0 0 0 309 0 0 0 25 0 1 0 528892073 467308544 446 33554432000 134512640 134534564 4287614208 18446744073709551615 134528275 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10816/statm: 114089 446 95 6 0 112755 0
[pid=10817] ppid=10813 vsize=21848 CPUtime=3.09 cores=3
/proc/10817/stat : 10817 (march_hi) R 10813 10813 10103 0 -1 4202496 6081 0 0 0 308 1 0 0 25 0 1 0 528892073 22372352 3899 33554432000 4194304 4331988 140735581680816 18446744073709551615 4239602 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10817/statm: 5462 3899 174 34 0 3904 0
[pid=10818] ppid=10813 vsize=128416 CPUtime=12.29 cores=4-7
/proc/10818/stat : 10818 (plingeling) S 10813 10813 10103 0 -1 4202496 5555 1249 0 0 1229 0 0 0 18 0 5 0 528892073 131497984 4704 33554432000 4194304 4356188 140737295992784 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/10818/statm: 32104 4704 164 40 0 30648 0
[pid=10818/tid=10825] ppid=10813 vsize=128416 CPUtime=3.08 cores=4-7
/proc/10818/task/10825/stat : 10825 (plingeling) R 10813 10813 10103 0 -1 4202560 1395 1249 0 0 308 0 0 0 25 0 5 0 528892074 131497984 4704 33554432000 4194304 4356188 140737295992784 18446744073709551615 4279783 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10818/tid=10826] ppid=10813 vsize=128416 CPUtime=3.07 cores=4-7
/proc/10818/task/10826/stat : 10826 (plingeling) R 10813 10813 10103 0 -1 4202560 1062 1249 0 0 307 0 0 0 25 0 5 0 528892074 131497984 4704 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331309 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10818/tid=10827] ppid=10813 vsize=128416 CPUtime=3.07 cores=4-7
/proc/10818/task/10827/stat : 10827 (plingeling) R 10813 10813 10103 0 -1 4202560 969 1249 0 0 307 0 0 0 25 0 5 0 528892074 131497984 4704 33554432000 4194304 4356188 140737295992784 18446744073709551615 4332239 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10818/tid=10828] ppid=10813 vsize=128416 CPUtime=3.05 cores=4-7
/proc/10818/task/10828/stat : 10828 (plingeling) R 10813 10813 10103 0 -1 4202560 1143 1249 0 0 305 0 0 0 25 0 5 0 528892074 131497984 4704 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331086 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.65
Current children cumulated vsize (KiB) 2533660

[startup+6.30103 s]
/proc/loadavg: 8.23 8.21 8.14 9/184 10828
/proc/meminfo: memFree=30760644/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 7 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
[pid=10814] ppid=10813 vsize=1900136 CPUtime=6.29 cores=0
/proc/10814/stat : 10814 (cryptominisat) R 10813 10813 10103 0 -1 4202496 6151 0 0 0 628 1 0 0 25 0 1 0 528892073 1945739264 3239 33554432000 4194304 4587354 140735243217040 18446744073709551615 4238332 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10814/statm: 475034 3239 338 96 0 471596 0
[pid=10815] ppid=10813 vsize=15848 CPUtime=6.29 cores=1
/proc/10815/stat : 10815 (clasp) R 10813 10813 10103 0 -1 4202496 1394 0 0 0 629 0 0 0 25 0 1 0 528892073 16228352 1280 33554432000 4194304 4785740 140736260543024 18446744073709551615 4287249 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10815/statm: 3962 1280 340 145 0 1007 0
[pid=10816] ppid=10813 vsize=456356 CPUtime=6.29 cores=2
/proc/10816/stat : 10816 (TNM) R 10813 10813 10103 0 -1 4202496 549 0 0 0 629 0 0 0 25 0 1 0 528892073 467308544 446 33554432000 134512640 134534564 4287614208 18446744073709551615 134528842 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10816/statm: 114089 446 95 6 0 112755 0
[pid=10817] ppid=10813 vsize=21848 CPUtime=6.29 cores=3
/proc/10817/stat : 10817 (march_hi) R 10813 10813 10103 0 -1 4202496 6123 0 0 0 628 1 0 0 25 0 1 0 528892073 22372352 3941 33554432000 4194304 4331988 140735581680816 18446744073709551615 4228224 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10817/statm: 5462 3941 174 34 0 3904 0
[pid=10818] ppid=10813 vsize=129168 CPUtime=25.08 cores=4-7
/proc/10818/stat : 10818 (plingeling) S 10813 10813 10103 0 -1 4202496 5981 1249 0 0 2508 0 0 0 18 0 5 0 528892073 132268032 5084 33554432000 4194304 4356188 140737295992784 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/10818/statm: 32292 5084 164 40 0 30836 0
[pid=10818/tid=10825] ppid=10813 vsize=129168 CPUtime=6.28 cores=4-7
/proc/10818/task/10825/stat : 10825 (plingeling) R 10813 10813 10103 0 -1 4202560 1680 1249 0 0 628 0 0 0 25 0 5 0 528892074 132268032 5084 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10818/tid=10826] ppid=10813 vsize=129168 CPUtime=6.27 cores=4-7
/proc/10818/task/10826/stat : 10826 (plingeling) R 10813 10813 10103 0 -1 4202560 1062 1249 0 0 627 0 0 0 25 0 5 0 528892074 132268032 5084 33554432000 4194304 4356188 140737295992784 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10818/tid=10827] ppid=10813 vsize=129168 CPUtime=6.27 cores=4-7
/proc/10818/task/10827/stat : 10827 (plingeling) R 10813 10813 10103 0 -1 4202560 1031 1249 0 0 627 0 0 0 25 0 5 0 528892074 132268032 5084 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331984 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10818/tid=10828] ppid=10813 vsize=129168 CPUtime=6.23 cores=4-7
/proc/10818/task/10828/stat : 10828 (plingeling) R 10813 10813 10103 0 -1 4202560 1222 1249 0 0 623 0 0 0 25 0 5 0 528892074 132268032 5084 33554432000 4194304 4356188 140737295992784 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.24
Current children cumulated vsize (KiB) 2541680

[startup+12.7009 s]
/proc/loadavg: 8.19 8.21 8.14 9/184 10828
/proc/meminfo: memFree=30756084/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 7 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
[pid=10814] ppid=10813 vsize=1904776 CPUtime=12.69 cores=0
/proc/10814/stat : 10814 (cryptominisat) R 10813 10813 10103 0 -1 4202496 8768 0 0 0 1267 2 0 0 25 0 1 0 528892073 1950490624 3333 33554432000 4194304 4587354 140735243217040 18446744073709551615 4238421 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10814/statm: 476194 3333 338 96 0 472756 0
[pid=10815] ppid=10813 vsize=16680 CPUtime=12.69 cores=1
/proc/10815/stat : 10815 (clasp) R 10813 10813 10103 0 -1 4202496 1619 0 0 0 1269 0 0 0 25 0 1 0 528892073 17080320 1505 33554432000 4194304 4785740 140736260543024 18446744073709551615 4553088 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10815/statm: 4170 1505 341 145 0 1215 0
[pid=10816] ppid=10813 vsize=456356 CPUtime=12.69 cores=2
/proc/10816/stat : 10816 (TNM) R 10813 10813 10103 0 -1 4202496 549 0 0 0 1269 0 0 0 25 0 1 0 528892073 467308544 446 33554432000 134512640 134534564 4287614208 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10816/statm: 114089 446 95 6 0 112755 0
[pid=10817] ppid=10813 vsize=22412 CPUtime=12.69 cores=3
/proc/10817/stat : 10817 (march_hi) R 10813 10813 10103 0 -1 4202496 6165 0 0 0 1268 1 0 0 25 0 1 0 528892073 22949888 3983 33554432000 4194304 4331988 140735581680816 18446744073709551615 4239610 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10817/statm: 5603 3983 174 34 0 4045 0
[pid=10818] ppid=10813 vsize=132316 CPUtime=50.65 cores=4-7
/proc/10818/stat : 10818 (plingeling) S 10813 10813 10103 0 -1 4202496 8052 1249 0 0 5064 1 0 0 18 0 5 0 528892073 135491584 5935 33554432000 4194304 4356188 140737295992784 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/10818/statm: 33079 5935 164 40 0 31623 0
[pid=10818/tid=10825] ppid=10813 vsize=132316 CPUtime=12.68 cores=4-7
/proc/10818/task/10825/stat : 10825 (plingeling) R 10813 10813 10103 0 -1 4202560 2001 1249 0 0 1268 0 0 0 25 0 5 0 528892074 135491584 5935 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331141 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10818/tid=10826] ppid=10813 vsize=132316 CPUtime=12.67 cores=4-7
/proc/10818/task/10826/stat : 10826 (plingeling) R 10813 10813 10103 0 -1 4202560 1705 1249 0 0 1267 0 0 0 25 0 5 0 528892074 135491584 5935 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331076 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10818/tid=10827] ppid=10813 vsize=132316 CPUtime=12.67 cores=4-7
/proc/10818/task/10827/stat : 10827 (plingeling) R 10813 10813 10103 0 -1 4202560 1449 1249 0 0 1267 0 0 0 25 0 5 0 528892074 135491584 5935 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10818/tid=10828] ppid=10813 vsize=132316 CPUtime=12.59 cores=4-7
/proc/10818/task/10828/stat : 10828 (plingeling) R 10813 10813 10103 0 -1 4202560 1911 1249 0 0 1259 0 0 0 25 0 5 0 528892074 135491584 5935 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.41
Current children cumulated vsize (KiB) 2550864

[startup+25.5006 s]
/proc/loadavg: 8.16 8.20 8.13 9/184 10828
/proc/meminfo: memFree=30748568/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 3 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
[pid=10814] ppid=10813 vsize=1900716 CPUtime=25.48 cores=0
/proc/10814/stat : 10814 (cryptominisat) R 10813 10813 10103 0 -1 4202496 12775 0 0 0 2546 2 0 0 25 0 1 0 528892073 1946333184 3573 33554432000 4194304 4587354 140735243217040 18446744073709551615 4238736 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=10816] ppid=10813 vsize=456360 CPUtime=4841.96 cores=2
/proc/10816/stat : 10816 (TNM) R 10813 10813 10103 0 -1 4202496 558 0 0 0 484193 3 0 0 25 0 1 0 528892073 467312640 455 33554432000 134512640 134534564 4287614208 18446744073709551615 134530598 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10816/statm: 114090 455 103 6 0 112756 0
[pid=10817] ppid=10813 vsize=170292 CPUtime=4842.42 cores=3
/proc/10817/stat : 10817 (march_hi) R 10813 10813 10103 0 -1 4202496 34647 0 0 0 484222 20 0 0 25 0 1 0 528892073 174379008 32465 33554432000 4194304 4331988 140735581680816 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10817/statm: 42573 32465 174 34 0 41015 0
[pid=10818] ppid=10813 vsize=309240 CPUtime=19342.4 cores=4-7
/proc/10818/stat : 10818 (plingeling) S 10813 10813 10103 0 -1 4202496 1048248 1249 0 0 1934079 165 0 0 18 0 5 0 528892073 316661760 49925 33554432000 4194304 4356188 140737295992784 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/10818/statm: 77310 49925 164 40 0 75854 0
[pid=10818/tid=10825] ppid=10813 vsize=309240 CPUtime=4842.42 cores=4-7
/proc/10818/task/10825/stat : 10825 (plingeling) R 10813 10813 10103 0 -1 4202560 263340 1249 0 0 484201 41 0 0 25 0 5 0 528892074 316661760 49925 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10818/tid=10826] ppid=10813 vsize=309240 CPUtime=4842.39 cores=4-7
/proc/10818/task/10826/stat : 10826 (plingeling) R 10813 10813 10103 0 -1 4202560 281584 1249 0 0 484192 47 0 0 25 0 5 0 528892074 316661760 49925 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10818/tid=10827] ppid=10813 vsize=309240 CPUtime=4842.41 cores=4-7
/proc/10818/task/10827/stat : 10827 (plingeling) R 10813 10813 10103 0 -1 4202560 271186 1249 0 0 484202 39 0 0 25 0 5 0 528892074 316661760 49925 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10818/tid=10828] ppid=10813 vsize=309240 CPUtime=4815.18 cores=4-7
/proc/10818/task/10828/stat : 10828 (plingeling) R 10813 10813 10103 0 -1 4202560 231152 1249 0 0 481481 37 0 0 25 0 5 0 528892074 316661760 49925 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38711.5
Current children cumulated vsize (KiB) 2991300

[startup+4902.3 s]
/proc/loadavg: 8.00 8.05 8.07 9/184 10970
/proc/meminfo: memFree=30325604/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 3 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
[pid=10814] ppid=10813 vsize=1995572 CPUtime=4902.28 cores=0
/proc/10814/stat : 10814 (cryptominisat) R 10813 10813 10103 0 -1 4202496 356202 0 0 0 490099 129 0 0 25 0 1 0 528892073 2043465728 28906 33554432000 4194304 4587354 140735243217040 18446744073709551615 4238406 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10814/statm: 498893 28906 339 96 0 495455 0
[pid=10815] ppid=10813 vsize=40292 CPUtime=4902.43 cores=1
/proc/10815/stat : 10815 (clasp) R 10813 10813 10103 0 -1 4202496 7513 0 0 0 490241 2 0 0 25 0 1 0 528892073 41259008 6892 33554432000 4194304 4785740 140736260543024 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10815/statm: 10073 6892 341 145 0 7118 0
[pid=10816] ppid=10813 vsize=456360 CPUtime=4901.95 cores=2
/proc/10816/stat : 10816 (TNM) R 10813 10813 10103 0 -1 4202496 558 0 0 0 490192 3 0 0 25 0 1 0 528892073 467312640 455 33554432000 134512640 134534564 4287614208 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10816/statm: 114090 455 103 6 0 112756 0
[pid=10817] ppid=10813 vsize=170292 CPUtime=4902.43 cores=3
/proc/10817/stat : 10817 (march_hi) R 10813 10813 10103 0 -1 4202496 34965 0 0 0 490222 21 0 0 25 0 1 0 528892073 174379008 32783 33554432000 4194304 4331988 140735581680816 18446744073709551615 4298853 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10817/statm: 42573 32783 174 34 0 41015 0
[pid=10818] ppid=10813 vsize=324216 CPUtime=19582.1 cores=4-7
/proc/10818/stat : 10818 (plingeling) S 10813 10813 10103 0 -1 4202496 1058392 1249 0 0 1958046 167 0 0 18 0 5 0 528892073 331997184 53669 33554432000 4194304 4356188 140737295992784 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/10818/statm: 81054 53669 164 40 0 79598 0
[pid=10818/tid=10825] ppid=10813 vsize=324216 CPUtime=4902.42 cores=4-7
/proc/10818/task/10825/stat : 10825 (plingeling) R 10813 10813 10103 0 -1 4202560 264748 1249 0 0 490201 41 0 0 25 0 5 0 528892074 331997184 53669 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331992 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10818/tid=10826] ppid=10813 vsize=324216 CPUtime=4902.4 cores=4-7
/proc/10818/task/10826/stat : 10826 (plingeling) R 10813 10813 10103 0 -1 4202560 287472 1249 0 0 490192 48 0 0 25 0 5 0 528892074 331997184 53669 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10818/tid=10827] ppid=10813 vsize=324216 CPUtime=4902.41 cores=4-7
/proc/10818/task/10827/stat : 10827 (plingeling) R 10813 10813 10103 0 -1 4202560 272594 1249 0 0 490202 39 0 0 25 0 5 0 528892074 331997184 53669 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10818/tid=10828] ppid=10813 vsize=324216 CPUtime=4874.86 cores=4-7
/proc/10818/task/10828/stat : 10828 (plingeling) R 10813 10813 10103 0 -1 4202560 232592 1249 0 0 487449 37 0 0 25 0 5 0 528892074 331997184 53669 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331151 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39191.2
Current children cumulated vsize (KiB) 3005056

[startup+4962.3 s]
/proc/loadavg: 8.00 8.04 8.06 9/184 10972
/proc/meminfo: memFree=30309072/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 3 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
[pid=10814] ppid=10813 vsize=1995572 CPUtime=4962.28 cores=0
/proc/10814/stat : 10814 (cryptominisat) R 10813 10813 10103 0 -1 4202496 357018 0 0 0 496099 129 0 0 25 0 1 0 528892073 2043465728 29691 33554432000 4194304 4587354 140735243217040 18446744073709551615 4232215 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10814/statm: 498893 29691 339 96 0 495455 0
[pid=10815] ppid=10813 vsize=40384 CPUtime=4962.43 cores=1
/proc/10815/stat : 10815 (clasp) R 10813 10813 10103 0 -1 4202496 7583 0 0 0 496241 2 0 0 25 0 1 0 528892073 41353216 6914 33554432000 4194304 4785740 140736260543024 18446744073709551615 4277424 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10815/statm: 10096 6914 341 145 0 7141 0
[pid=10816] ppid=10813 vsize=456360 CPUtime=4961.95 cores=2
/proc/10816/stat : 10816 (TNM) R 10813 10813 10103 0 -1 4202496 558 0 0 0 496192 3 0 0 25 0 1 0 528892073 467312640 455 33554432000 134512640 134534564 4287614208 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10816/statm: 114090 455 103 6 0 112756 0
[pid=10817] ppid=10813 vsize=170292 CPUtime=4962.43 cores=3
/proc/10817/stat : 10817 (march_hi) R 10813 10813 10103 0 -1 4202496 35263 0 0 0 496222 21 0 0 25 0 1 0 528892073 174379008 33081 33554432000 4194304 4331988 140735581680816 18446744073709551615 4210448 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10817/statm: 42573 33081 174 34 0 41015 0
[pid=10818] ppid=10813 vsize=334840 CPUtime=19821.8 cores=4-7
/proc/10818/stat : 10818 (plingeling) S 10813 10813 10103 0 -1 4202496 1068440 1249 0 0 1982012 168 0 0 18 0 5 0 528892073 342876160 56325 33554432000 4194304 4356188 140737295992784 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/10818/statm: 83710 56325 164 40 0 82254 0
[pid=10818/tid=10825] ppid=10813 vsize=334840 CPUtime=4962.42 cores=4-7
/proc/10818/task/10825/stat : 10825 (plingeling) R 10813 10813 10103 0 -1 4202560 267692 1249 0 0 496201 41 0 0 25 0 5 0 528892074 342876160 56325 33554432000 4194304 4356188 140737295992784 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10818/tid=10826] ppid=10813 vsize=334840 CPUtime=4962.39 cores=4-7
/proc/10818/task/10826/stat : 10826 (plingeling) R 10813 10813 10103 0 -1 4202560 288752 1249 0 0 496191 48 0 0 25 0 5 0 528892074 342876160 56325 33554432000 4194304 4356188 140737295992784 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10818/tid=10827] ppid=10813 vsize=334840 CPUtime=4962.41 cores=4-7
/proc/10818/task/10827/stat : 10827 (plingeling) R 10813 10813 10103 0 -1 4202560 273298 1249 0 0 496202 39 0 0 25 0 5 0 528892074 342876160 56325 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10818/tid=10828] ppid=10813 vsize=334840 CPUtime=4934.52 cores=4-7
/proc/10818/task/10828/stat : 10828 (plingeling) R 10813 10813 10103 0 -1 4202560 237712 1249 0 0 493415 37 0 0 25 0 5 0 528892074 342876160 56325 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331151 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39670.9
Current children cumulated vsize (KiB) 3015772



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.5 s]
/proc/loadavg: 8.00 8.03 8.06 9/184 10973
/proc/meminfo: memFree=30283244/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 3 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
[pid=10814] ppid=10813 vsize=2017132 CPUtime=5003.49 cores=0
/proc/10814/stat : 10814 (cryptominisat) R 10813 10813 10103 0 -1 4202496 361684 0 0 0 500218 131 0 0 25 0 1 0 528892073 2065543168 33471 33554432000 4194304 4587354 140735243217040 18446744073709551615 4238406 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10814/statm: 504283 33471 339 96 0 500845 0
[pid=10815] ppid=10813 vsize=40668 CPUtime=5003.64 cores=1
/proc/10815/stat : 10815 (clasp) R 10813 10813 10103 0 -1 4202496 7665 0 0 0 500362 2 0 0 25 0 1 0 528892073 41644032 6996 33554432000 4194304 4785740 140736260543024 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10815/statm: 10167 6996 341 145 0 7212 0
[pid=10816] ppid=10813 vsize=456360 CPUtime=5003.15 cores=2
/proc/10816/stat : 10816 (TNM) R 10813 10813 10103 0 -1 4202496 558 0 0 0 500312 3 0 0 25 0 1 0 528892073 467312640 455 33554432000 134512640 134534564 4287614208 18446744073709551615 134528383 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10816/statm: 114090 455 103 6 0 112756 0
[pid=10817] ppid=10813 vsize=170292 CPUtime=5003.64 cores=3
/proc/10817/stat : 10817 (march_hi) R 10813 10813 10103 0 -1 4202496 35457 0 0 0 500342 22 0 0 25 0 1 0 528892073 174379008 33275 33554432000 4194304 4331988 140735581680816 18446744073709551615 4210461 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10817/statm: 42573 33275 174 34 0 41015 0
[pid=10818] ppid=10813 vsize=345592 CPUtime=19986.4 cores=4-7
/proc/10818/stat : 10818 (plingeling) S 10813 10813 10103 0 -1 4202496 1073816 1249 0 0 1998471 169 0 0 18 0 5 0 528892073 353886208 59013 33554432000 4194304 4356188 140737295992784 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/10818/statm: 86398 59013 164 40 0 84942 0
[pid=10818/tid=10825] ppid=10813 vsize=345592 CPUtime=5003.63 cores=4-7
/proc/10818/task/10825/stat : 10825 (plingeling) R 10813 10813 10103 0 -1 4202560 267820 1249 0 0 500321 42 0 0 25 0 5 0 528892074 353886208 59013 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331076 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10818/tid=10826] ppid=10813 vsize=345592 CPUtime=5003.61 cores=4-7
/proc/10818/task/10826/stat : 10826 (plingeling) R 10813 10813 10103 0 -1 4202560 291440 1249 0 0 500312 49 0 0 25 0 5 0 528892074 353886208 59013 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10818/tid=10827] ppid=10813 vsize=345592 CPUtime=5003.62 cores=4-7
/proc/10818/task/10827/stat : 10827 (plingeling) R 10813 10813 10103 0 -1 4202560 275346 1249 0 0 500323 39 0 0 25 0 5 0 528892074 353886208 59013 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10818/tid=10828] ppid=10813 vsize=345592 CPUtime=4975.5 cores=4-7
/proc/10818/task/10828/stat : 10828 (plingeling) R 10813 10813 10103 0 -1 4202560 238224 1249 0 0 497513 37 0 0 25 0 5 0 528892074 353886208 59013 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 3048368

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5003.5 s]
/proc/loadavg: 8.00 8.03 8.06 9/184 10973
/proc/meminfo: memFree=30283244/32951124 swapFree=67111336/67111528
[pid=10813] ppid=10811 vsize=18324 CPUtime=0 cores=0-7
/proc/10813/stat : 10813 (ppfolio) S 10811 10813 10103 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 528892072 18763776 390 33554432000 4194304 4287625 140735710314256 18446744073709551615 271550557711 0 0 4096 65536 5630071421993484287 0 0 17 3 0 0 0
/proc/10813/statm: 4581 390 341 23 0 91 0
[pid=10814] ppid=10813 vsize=2017132 CPUtime=5003.49 cores=0
/proc/10814/stat : 10814 (cryptominisat) R 10813 10813 10103 0 -1 4202496 361684 0 0 0 500218 131 0 0 25 0 1 0 528892073 2065543168 33471 33554432000 4194304 4587354 140735243217040 18446744073709551615 4238406 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/10814/statm: 504283 33471 339 96 0 500845 0
[pid=10815] ppid=10813 vsize=40668 CPUtime=5003.64 cores=1
/proc/10815/stat : 10815 (clasp) R 10813 10813 10103 0 -1 4202496 7665 0 0 0 500362 2 0 0 25 0 1 0 528892073 41644032 6996 33554432000 4194304 4785740 140736260543024 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/10815/statm: 10167 6996 341 145 0 7212 0
[pid=10816] ppid=10813 vsize=456360 CPUtime=5003.15 cores=2
/proc/10816/stat : 10816 (TNM) R 10813 10813 10103 0 -1 4202496 558 0 0 0 500312 3 0 0 25 0 1 0 528892073 467312640 455 33554432000 134512640 134534564 4287614208 18446744073709551615 134528383 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10816/statm: 114090 455 103 6 0 112756 0
[pid=10817] ppid=10813 vsize=170292 CPUtime=5003.64 cores=3
/proc/10817/stat : 10817 (march_hi) R 10813 10813 10103 0 -1 4202496 35457 0 0 0 500342 22 0 0 25 0 1 0 528892073 174379008 33275 33554432000 4194304 4331988 140735581680816 18446744073709551615 4210461 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/10817/statm: 42573 33275 174 34 0 41015 0
[pid=10818] ppid=10813 vsize=345592 CPUtime=19986.4 cores=4-7
/proc/10818/stat : 10818 (plingeling) S 10813 10813 10103 0 -1 4202496 1073816 1249 0 0 1998471 169 0 0 18 0 5 0 528892073 353886208 59013 33554432000 4194304 4356188 140737295992784 18446744073709551615 271562340325 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/10818/statm: 86398 59013 164 40 0 84942 0
[pid=10818/tid=10825] ppid=10813 vsize=345592 CPUtime=5003.63 cores=4-7
/proc/10818/task/10825/stat : 10825 (plingeling) R 10813 10813 10103 0 -1 4202560 267820 1249 0 0 500321 42 0 0 25 0 5 0 528892074 353886208 59013 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331076 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=10818/tid=10826] ppid=10813 vsize=345592 CPUtime=5003.61 cores=4-7
/proc/10818/task/10826/stat : 10826 (plingeling) R 10813 10813 10103 0 -1 4202560 291440 1249 0 0 500312 49 0 0 25 0 5 0 528892074 353886208 59013 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=10818/tid=10827] ppid=10813 vsize=345592 CPUtime=5003.62 cores=4-7
/proc/10818/task/10827/stat : 10827 (plingeling) R 10813 10813 10103 0 -1 4202560 275346 1249 0 0 500323 39 0 0 25 0 5 0 528892074 353886208 59013 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=10818/tid=10828] ppid=10813 vsize=345592 CPUtime=4975.5 cores=4-7
/proc/10818/task/10828/stat : 10828 (plingeling) R 10813 10813 10103 0 -1 4202560 238224 1249 0 0 497513 37 0 0 25 0 5 0 528892074 353886208 59013 33554432000 4194304 4356188 140737295992784 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 40000.3
Current children cumulated vsize (KiB) 3048368

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 10813 and gives
#  childrusage.ru_utime.tv_sec=20012
#  childrusage.ru_utime.tv_usec=355659
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=620753
# CPU time returned by wait4() is 20014
# while last known CPU time is 40000.3
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5003.52
CPU time (s): 40000.3
CPU user time (s): 39997.1
CPU system time (s): 3.27
CPU usage (%): 799.444
Max. virtual memory (cumulated for all children) (KiB): 8548772

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 20012.4
system time used= 1.62075
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 405901
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 774
involuntary context switches= 245574

runsolver used 7.51086 second user time and 19.79 second system time

The end

Launcher Data 

Begin job on node133 at 2011-04-30 16:15:42
IDJOB=3276031
IDBENCH=84634
IDSOLVER=1638
FILE ID=node133/3276031-1304172942
RUNJOBID= node133-1304162563-10117
PBS_JOBID= 13170940
Free space on /tmp= 72756 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT04/handmade/connamacher/connm-d0.04/connm-ue-csp-sat-n800-d0.04-s1788334438.used-as.sat04-976.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3276031-1304172942/watcher-3276031-1304172942 -o /tmp/evaluation-result-3276031-1304172942/solver-3276031-1304172942 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3276031-1304172942.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 41d80cc808c021a52e21ee9950bc91e0
RANDOM SEED=1533300427

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30812580 kB
Buffers:        547960 kB
Cached:        1180756 kB
SwapCached:        124 kB
Active:         238468 kB
Inactive:      1553700 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30812580 kB
SwapTotal:    67111528 kB
SwapFree:     67111336 kB
Dirty:           10100 kB
Writeback:           0 kB
AnonPages:       64084 kB
Mapped:          14720 kB
Slab:           280604 kB
PageTables:       4744 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   184460 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72744 MiB
End job on node133 at 2011-04-30 17:39:07