Trace number 3188414

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
QuteRSat 2011-03-01? (TO) 1300.01 1300.04

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v30000-c126000-S629207485-001-UNKNOWN.cnf
MD5SUM33c09da85657b6bbed23989098d6baf7
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark50.0704
Satisfiable
(Un)Satisfiability was proved
Number of variables30000
Number of clauses126000
Sum of the clauses size378000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3126000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c Set FPU double precision.
0.09/0.15	c Parse Cnf success: #var: 30000 #clause: 126000
0.09/0.15	c Find Units :          0
0.09/0.15	c Find Invs  :          0
0.09/0.16	c Find Ands  :          0 AvgFaninNum: nan
0.09/0.17	c Find Xors  :          0 AvgFaninNum: nan
0.09/0.19	c Find Ite   :          0
0.19/0.25	c UnDirect variable: 0
0.19/0.25	c UnDirect to Direct Gate: 0
0.19/0.25	c Forced Direct Gate: 0 Unuse unDirect Gate: 0
0.19/0.25	c Break Cycle : 0
0.19/0.25	c Total Root : 0 Undef Var: 30000
0.19/0.25	c After covering, Total Root: 0 Leaf: 0
0.19/0.25	c Variable( cover/uncovered/unused ): 0 30000 0
0.19/0.25	c Unit Clauses: 0
0.19/0.25	c Binary Clauses( cover/uncovered ): 0 0
0.19/0.25	c Clauses( cover/uncovered ): 0 126000
0.19/0.25	c Total Clauses( cover/uncovered ): 0 126000 0%
0.19/0.25	c Covered Ands: 0 Invs: 0 Xors: 0 Ites: 0
0.29/0.37	c Uncover Clauses avg size: 3 max size: 3
0.29/0.38	c Final circuit: i/l/o/s   30000       0  126000    251994
0.29/0.38	c CnfToAig: Time: 0.37 s
0.49/0.58	c TmMap: Set choices finish.
0.59/0.68	c TmMap: Set big and: 126000
0.59/0.68	c TmMap: Origin: 281995 ClauseNum: 755982 --> New: 156001 ClauseNum: 504000
0.59/0.68	c TmMap: Time: 0.13
0.69/0.72	c Try to Add: #clause: 252000 #xorclause: 0
0.69/0.78	c #var: 30000 #clauses: 126001 Solving... 
1299.96/1300.00	c Time Out !!
1299.96/1300.00	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 893)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3188414-1302280459/watcher-3188414-1302280459 -o /tmp/evaluation-result-3188414-1302280459/solver-3188414-1302280459 -C 2400 -W 1300 -M 7680 qutersat_release HOME/instance-3188414-1302280459.cnf 537976258 

running on 2 cores: 1,3

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.92 3.98 4.15 4/189 4237
/proc/meminfo: memFree=17284148/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=33388 CPUtime=0 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 1971 0 0 0 0 0 0 0 22 0 1 0 339646857 34189312 1904 33554432000 4194304 6016207 140736406981152 18446744073709551615 4779602 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 8347 1904 378 445 0 1891 0

[startup+0.100803 s]
/proc/loadavg: 3.92 3.98 4.15 4/189 4237
/proc/meminfo: memFree=17284148/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=60140 CPUtime=0.09 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 8691 0 0 0 8 1 0 0 22 0 1 0 339646857 61583360 8623 33554432000 4194304 6016207 140736406981152 18446744073709551615 4305236 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 15035 8623 378 445 0 8579 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 60140

[startup+0.20079 s]
/proc/loadavg: 3.92 3.98 4.15 4/189 4237
/proc/meminfo: memFree=17284148/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=61164 CPUtime=0.19 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 8981 0 0 0 18 1 0 0 23 0 1 0 339646857 62631936 8913 33554432000 4194304 6016207 140736406981152 18446744073709551615 4809616 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 15291 8913 395 445 0 8835 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 61164

[startup+0.300774 s]
/proc/loadavg: 3.92 3.98 4.15 4/189 4237
/proc/meminfo: memFree=17284148/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=61164 CPUtime=0.29 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 9175 0 0 0 28 1 0 0 23 0 1 0 339646857 62631936 9107 33554432000 4194304 6016207 140736406981152 18446744073709551615 4264276 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 15291 9107 400 445 0 8835 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 61164

[startup+0.700701 s]
/proc/loadavg: 3.92 3.98 4.15 4/189 4237
/proc/meminfo: memFree=17284148/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=132012 CPUtime=0.69 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 26355 0 0 0 66 3 0 0 25 0 1 0 339646857 135180288 26287 33554432000 4194304 6016207 140736406981152 18446744073709551615 4809773 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 33003 26287 672 445 0 26547 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 132012

[startup+1.50056 s]
/proc/loadavg: 3.92 3.98 4.15 5/190 4238
/proc/meminfo: memFree=17129096/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=132012 CPUtime=1.49 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 26406 0 0 0 146 3 0 0 25 0 1 0 339646857 135180288 26338 33554432000 4194304 6016207 140736406981152 18446744073709551615 4378298 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 33003 26338 719 445 0 26547 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 132012

[startup+3.10028 s]
/proc/loadavg: 3.92 3.98 4.15 5/190 4238
/proc/meminfo: memFree=17118928/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=164828 CPUtime=3.09 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 26787 0 0 0 306 3 0 0 25 0 1 0 339646857 168783872 26719 33554432000 4194304 6016207 140736406981152 18446744073709551615 4378285 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 41207 26719 719 445 0 34751 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 164828

[startup+6.30071 s]
/proc/loadavg: 4.01 4.00 4.15 5/190 4238
/proc/meminfo: memFree=17066164/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=164828 CPUtime=6.29 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 29207 0 0 0 625 4 0 0 25 0 1 0 339646857 168783872 29139 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425705 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 41207 29139 719 445 0 34751 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 164828

[startup+12.7006 s]
/proc/loadavg: 4.01 4.00 4.15 5/190 4238
/proc/meminfo: memFree=16924288/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=239880 CPUtime=12.69 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 41899 0 0 0 1264 5 0 0 25 0 1 0 339646857 245637120 41831 33554432000 4194304 6016207 140736406981152 18446744073709551615 4377997 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 59970 41831 727 445 0 53514 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 239880

[startup+25.5003 s]
/proc/loadavg: 4.00 4.00 4.15 5/190 4238
/proc/meminfo: memFree=16812092/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=315980 CPUtime=25.49 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 48671 0 0 0 2541 8 0 0 25 0 1 0 339646857 323563520 48603 33554432000 4194304 6016207 140736406981152 18446744073709551615 4380063 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 78995 48603 727 445 0 72539 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 315980

[startup+51.1008 s]
/proc/loadavg: 4.06 4.01 4.15 5/190 4239
/proc/meminfo: memFree=16626012/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=378272 CPUtime=51.1 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 70957 0 0 0 5098 12 0 0 25 0 1 0 339646857 387350528 70859 33554432000 4194304 6016207 140736406981152 18446744073709551615 4379459 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 94568 70859 727 445 0 88112 0
Current children cumulated CPU time (s) 51.1
Current children cumulated vsize (KiB) 378272

[startup+102.301 s]
/proc/loadavg: 4.02 4.01 4.14 5/190 4240
/proc/meminfo: memFree=16411612/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=418144 CPUtime=102.29 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 87300 0 0 0 10215 14 0 0 25 0 1 0 339646857 428179456 87047 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425113 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 104536 87047 727 445 0 98080 0
Current children cumulated CPU time (s) 102.29
Current children cumulated vsize (KiB) 418144

[startup+162.301 s]
/proc/loadavg: 4.01 4.00 4.13 5/190 4242
/proc/meminfo: memFree=16327884/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=432712 CPUtime=162.29 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 92936 0 0 0 16213 16 0 0 25 0 1 0 339646857 443097088 92594 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425254 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 108178 92594 727 445 0 101722 0
Current children cumulated CPU time (s) 162.29
Current children cumulated vsize (KiB) 432712

[startup+222.307 s]
/proc/loadavg: 4.00 4.00 4.12 5/190 4243
/proc/meminfo: memFree=16291524/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=548508 CPUtime=222.3 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 100409 0 0 0 22213 17 0 0 25 0 1 0 339646857 561672192 99909 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425137 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 137127 99909 727 445 0 130671 0
Current children cumulated CPU time (s) 222.3
Current children cumulated vsize (KiB) 548508

[startup+282.301 s]
/proc/loadavg: 4.00 4.00 4.10 5/190 4245
/proc/meminfo: memFree=16231728/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=548508 CPUtime=282.28 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 111506 0 0 0 28209 19 0 0 25 0 1 0 339646857 561672192 111006 33554432000 4194304 6016207 140736406981152 18446744073709551615 4380460 0 0 4096 16386 0 0 0 17 3 0 0 0
/proc/4237/statm: 137127 111006 727 445 0 130671 0
Current children cumulated CPU time (s) 282.28
Current children cumulated vsize (KiB) 548508

[startup+342.3 s]
/proc/loadavg: 4.04 4.01 4.10 5/190 4247
/proc/meminfo: memFree=16066308/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=606908 CPUtime=342.28 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 121910 0 0 0 34206 22 0 0 25 0 1 0 339646857 621473792 121274 33554432000 4194304 6016207 140736406981152 18446744073709551615 4424226 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 151727 121274 727 445 0 145271 0
Current children cumulated CPU time (s) 342.28
Current children cumulated vsize (KiB) 606908

[startup+402.301 s]
/proc/loadavg: 4.01 4.01 4.09 5/190 4248
/proc/meminfo: memFree=16022388/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=607036 CPUtime=402.28 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 123582 0 0 0 40205 23 0 0 25 0 1 0 339646857 621604864 122946 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425824 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 151759 122946 727 445 0 145303 0
Current children cumulated CPU time (s) 402.28
Current children cumulated vsize (KiB) 607036

[startup+462.301 s]
/proc/loadavg: 4.00 4.00 4.08 5/190 4250
/proc/meminfo: memFree=15982800/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=607036 CPUtime=462.28 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 124934 0 0 0 46205 23 0 0 25 0 1 0 339646857 621604864 124101 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425476 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 151759 124101 727 445 0 145303 0
Current children cumulated CPU time (s) 462.28
Current children cumulated vsize (KiB) 607036

[startup+522.301 s]
/proc/loadavg: 4.00 4.00 4.08 5/190 4251
/proc/meminfo: memFree=15881712/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=607036 CPUtime=522.28 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 132062 0 0 0 52203 25 0 0 25 0 1 0 339646857 621604864 131229 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425530 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 151759 131229 727 445 0 145303 0
Current children cumulated CPU time (s) 522.28
Current children cumulated vsize (KiB) 607036

[startup+582.301 s]
/proc/loadavg: 4.00 4.00 4.07 5/190 4253
/proc/meminfo: memFree=15809016/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=607036 CPUtime=582.27 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 132955 0 0 0 58202 25 0 0 25 0 1 0 339646857 621604864 132122 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425392 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 151759 132122 727 445 0 145303 0
Current children cumulated CPU time (s) 582.27
Current children cumulated vsize (KiB) 607036

[startup+642.3 s]
/proc/loadavg: 4.00 4.00 4.06 5/190 4255
/proc/meminfo: memFree=15760896/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=607036 CPUtime=642.28 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 133805 0 0 0 64202 26 0 0 25 0 1 0 339646857 621604864 132972 33554432000 4194304 6016207 140736406981152 18446744073709551615 4381469 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 151759 132972 727 445 0 145303 0
Current children cumulated CPU time (s) 642.28
Current children cumulated vsize (KiB) 607036

[startup+702.301 s]
/proc/loadavg: 4.00 4.00 4.06 5/190 4256
/proc/meminfo: memFree=15795212/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=607036 CPUtime=702.27 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 134973 0 0 0 70201 26 0 0 25 0 1 0 339646857 621604864 123099 33554432000 4194304 6016207 140736406981152 18446744073709551615 4426996 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 151759 123099 727 445 0 145303 0
Current children cumulated CPU time (s) 702.27
Current children cumulated vsize (KiB) 607036

[startup+762.301 s]
/proc/loadavg: 4.00 4.00 4.05 5/190 4258
/proc/meminfo: memFree=15782664/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=607036 CPUtime=762.27 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 136122 0 0 0 76201 26 0 0 25 0 1 0 339646857 621604864 124248 33554432000 4194304 6016207 140736406981152 18446744073709551615 4429764 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 151759 124248 727 445 0 145303 0
Current children cumulated CPU time (s) 762.27
Current children cumulated vsize (KiB) 607036

[startup+822.3 s]
/proc/loadavg: 4.00 4.00 4.05 5/190 4259
/proc/meminfo: memFree=15735760/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=607036 CPUtime=822.27 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 136600 0 0 0 82200 27 0 0 25 0 1 0 339646857 621604864 124726 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425389 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 151759 124726 727 445 0 145303 0
Current children cumulated CPU time (s) 822.27
Current children cumulated vsize (KiB) 607036

[startup+882.307 s]
/proc/loadavg: 4.05 4.01 4.05 5/190 4261
/proc/meminfo: memFree=15643232/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=607036 CPUtime=882.28 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 137101 0 0 0 88201 27 0 0 25 0 1 0 339646857 621604864 125227 33554432000 4194304 6016207 140736406981152 18446744073709551615 4378319 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 151759 125227 727 445 0 145303 0
Current children cumulated CPU time (s) 882.28
Current children cumulated vsize (KiB) 607036

[startup+942.3 s]
/proc/loadavg: 4.05 4.02 4.05 6/190 4263
/proc/meminfo: memFree=15556540/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=607548 CPUtime=942.27 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 149496 0 0 0 94198 29 0 0 25 0 1 0 339646857 622129152 137471 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425467 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 151887 137471 727 445 0 145431 0
Current children cumulated CPU time (s) 942.27
Current children cumulated vsize (KiB) 607548

[startup+1002.3 s]
/proc/loadavg: 4.02 4.02 4.04 5/190 4264
/proc/meminfo: memFree=15365804/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=732448 CPUtime=1002.27 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 182800 0 0 0 100189 38 0 0 25 0 1 0 339646857 750026752 170775 33554432000 4194304 6016207 140736406981152 18446744073709551615 4373888 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 183112 170776 727 445 0 176656 0
Current children cumulated CPU time (s) 1002.27
Current children cumulated vsize (KiB) 732448

[startup+1062.3 s]
/proc/loadavg: 4.00 4.01 4.03 5/190 4266
/proc/meminfo: memFree=15292364/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=794848 CPUtime=1062.27 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 186335 0 0 0 106187 40 0 0 25 0 1 0 339646857 813924352 174310 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425742 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 198712 174310 727 445 0 192256 0
Current children cumulated CPU time (s) 1062.27
Current children cumulated vsize (KiB) 794848

[startup+1122.3 s]
/proc/loadavg: 4.00 4.01 4.03 5/190 4267
/proc/meminfo: memFree=15226124/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=1046460 CPUtime=1122.26 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 190853 0 0 0 112185 41 0 0 25 0 1 0 339646857 1071575040 178828 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425742 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 261615 178828 727 445 0 255159 0
Current children cumulated CPU time (s) 1122.26
Current children cumulated vsize (KiB) 1046460

[startup+1182.3 s]
/proc/loadavg: 4.00 4.00 4.02 5/190 4269
/proc/meminfo: memFree=15162108/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=1298416 CPUtime=1182.27 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 203769 0 0 0 118182 45 0 0 25 0 1 0 339646857 1329577984 191744 33554432000 4194304 6016207 140736406981152 18446744073709551615 4379469 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 324604 191744 727 445 0 318148 0
Current children cumulated CPU time (s) 1182.27
Current children cumulated vsize (KiB) 1298416

[startup+1242.3 s]
/proc/loadavg: 4.00 4.00 4.02 5/190 4271
/proc/meminfo: memFree=15143248/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=1298416 CPUtime=1242.26 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 207050 0 0 0 124180 46 0 0 25 0 1 0 339646857 1329577984 195025 33554432000 4194304 6016207 140736406981152 18446744073709551615 4425530 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 324604 195025 727 445 0 318148 0
Current children cumulated CPU time (s) 1242.26
Current children cumulated vsize (KiB) 1298416



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 4.00 4.00 4.01 2/185 4275
/proc/meminfo: memFree=16610420/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=1298416 CPUtime=1299.96 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 207135 0 0 0 129950 46 0 0 25 0 1 0 339646857 1329577984 195110 33554432000 4194304 6016207 140736406981152 18446744073709551615 4418848 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 324604 195110 727 445 0 318148 0
Current children cumulated CPU time (s) 1299.96
Current children cumulated vsize (KiB) 1298416

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1300 s]
/proc/loadavg: 4.00 4.00 4.01 2/185 4275
/proc/meminfo: memFree=16610420/32951124 swapFree=67111528/67111528
[pid=4237] ppid=4235 vsize=1298416 CPUtime=1299.96 cores=1,3
/proc/4237/stat : 4237 (qutersat_releas) R 4235 4237 4177 0 -1 4202496 207135 0 0 0 129950 46 0 0 25 0 1 0 339646857 1329577984 195110 33554432000 4194304 6016207 140736406981152 18446744073709551615 4418848 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/4237/statm: 324604 195110 727 445 0 318148 0
Current children cumulated CPU time (s) 1299.96
Current children cumulated vsize (KiB) 1298416

Child status: 1
Real time (s): 1300.04
CPU time (s): 1300.01
CPU user time (s): 1299.51
CPU system time (s): 0.496924
CPU usage (%): 99.9975
Max. virtual memory (cumulated for all children) (KiB): 1298416

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1299.51
system time used= 0.496924
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 207140
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6
involuntary context switches= 3541

runsolver used 2.04669 second user time and 5.05123 second system time

The end

Launcher Data 

Begin job on node108 at 2011-04-08 18:34:19
IDJOB=3188414
IDBENCH=82545
IDSOLVER=1607
FILE ID=node108/3188414-1302280459
RUNJOBID= node108-1302280459-4197
PBS_JOBID= 12919073
Free space on /tmp= 71624 MiB

SOLVER NAME= QuteRSat 2011-03-01
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v30000-c126000-S629207485-001-UNKNOWN.cnf
COMMAND LINE= qutersat_release BENCHNAME RANDOMSEED
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3188414-1302280459/watcher-3188414-1302280459 -o /tmp/evaluation-result-3188414-1302280459/solver-3188414-1302280459 -C 2400 -W 1300 -M 7680  qutersat_release HOME/instance-3188414-1302280459.cnf 537976258

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= 33c09da85657b6bbed23989098d6baf7
RANDOM SEED=537976258

node108.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17284428 kB
Buffers:        584940 kB
Cached:       14528312 kB
SwapCached:          0 kB
Active:        8554904 kB
Inactive:      6730988 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17284428 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           11672 kB
Writeback:           0 kB
AnonPages:      172516 kB
Mapped:          18212 kB
Slab:           313496 kB
PageTables:       6200 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   363848 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71624 MiB
End job on node108 at 2011-04-08 18:56:01