Trace number 1779973

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth-5-0 2009-03-19 (4 threads)? (MO) 1977.9 502.838

General information on the benchmark

NameAPPLICATIONS/bioinfo/
q_query_3_l41_lambda.cnf
MD5SUM50760ceab82aaa514e253121456f71f6
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark33.89
Satisfiable
(Un)Satisfiability was proved
Number of variables29245
Number of clauses144673
Sum of the clauses size416481
Maximum clause length9
Minimum clause length1
Number of clauses of size 13878
Number of clauses of size 236799
Number of clauses of size 383760
Number of clauses of size 413947
Number of clauses of size 56166
Number of clauses of size over 5123

Solver Data 

0.31/0.34	c Command: HOME/ttsth-5-0 -num-threads 4 HOME/instance-1779973-1243680013.cnf 
0.31/0.34	c Using SATTIMEOUT of 10000
0.31/0.34	c Using SATRAM of 2047
0.31/0.34	c Size of subset node = 16
0.31/0.34	c Variables = 29245
0.31/0.34	c Clauses = 144673
0.31/0.34	c Literals = 416481
0.31/0.34	c Reduced size:
0.31/0.34	c Variables = 29245
0.31/0.34	c Clauses = 129790
0.31/0.34	c Literals = 379098
0.31/0.34	c a2
0.61/0.66	c a1
8.11/8.18	c Independent sub-problems = 1
8.11/8.18	c Sub-problem with 24459 variables and 129790 clauses
8.11/8.18	c using permutation method 4
8.11/8.18	c count = 1
8.11/8.18	c Ordering finished 8.130 seconds
8.11/8.18	c mapped 8.190 seconds
8.11/8.18	c permuted 8.190 seconds
8.11/8.18	c about to sort
8.21/8.23	c sorted
10.11/10.11	c clauses now 129790
10.11/10.11	c init finished 8.400 seconds
10.11/10.11	c tree built 10.120 seconds
10.11/10.11	c 
10.11/10.11	c PA progress                      | CPU progress Partial assignments | improvements      CPU time        PA/sec        nodes
10.11/10.11	c ---------------------------------|----------------------------------|------------------------------------------------------
10.11/10.11	c ................................ | ..........                       |                       10.1           0.0             
10.11/10.11	c ................................ | ..........                     1 |                       10.1           0.1             
10.11/10.11	c ****************................ | ..........                     2 |                       10.1           0.2            1
10.11/10.11	c ****************................ | ..........                     3 |                       10.1           0.3            1
10.11/10.11	c ****************................ | ..........                     4 |                       10.1           0.4            1
10.11/10.11	c ****************................ | ..........                     5 |                       10.1           0.5            1
10.11/10.11	c ****************................ | ..........                     6 |                       10.1           0.6            1
10.11/10.15	c ****************................ | ..........                  1000 | 2                     10.2          98.1          879
200.17/57.80	c ****************................ | ..........               1932000 | 56483                200.2        9651.3     16914004
399.78/107.80	c ****************................ | ..........               2928000 | 87300                400.1        7317.4     28075013
600.07/157.83	c ****************................ | ..........               3994000 | 119243               600.2        6654.7     38108623
799.82/207.87	c ****************................ | ..........               4863000 | 143104               800.1        6078.3     48295451
999.94/257.93	c ****************................ | *.........               5892000 | 171935              1000.0        5891.9     57047438
1199.84/308.07	c ****************................ | *.........               6899000 | 205769              1200.1        5748.6     72212833
1399.91/358.13	c ****************................ | *.........               7832000 | 233550              1400.0        5594.2     83300831
1599.70/408.21	c ****************................ | *.........               9075000 | 250902              1600.1        5671.6     92903026
1799.66/458.19	c ****************................ | *.........              10248000 | 282539              1800.0        5693.2    105978475

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver version 3.2.9 (svn:503) (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w /tmp/evaluation-result-1779973-1243680013/watcher-1779973-1243680013 -o /tmp/evaluation-result-1779973-1243680013/solver-1779973-1243680013 -C 10000 -W 1000 -M 2047 HOME/ttsth-5-0 -num-threads 4 HOME/instance-1779973-1243680013.cnf 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 10000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 10030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 2096128 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 2147328 KiB
Current StackSize limit: 8192 KiB


[startup+0 s]
/proc/loadavg: 4.09 4.10 3.74 3/99 16742
/proc/meminfo: memFree=3046168/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=1140 CPUtime=0.01
/proc/16742/stat : 16742 (ttsth-5-0) R 16740 16742 20984 0 -1 4202496 174 0 0 0 1 0 0 0 18 0 1 0 1132665695 1167360 130 4294967295 134512640 135237372 3218452640 3218452088 134732614 0 0 0 0 0 0 0 17 1 0 0 0
/proc/16742/statm: 285 130 75 177 0 107 0

[startup+0.051043 s]
/proc/loadavg: 4.09 4.10 3.74 3/99 16742
/proc/meminfo: memFree=3046168/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=3180 CPUtime=0.05
/proc/16742/stat : 16742 (ttsth-5-0) R 16740 16742 20984 0 -1 4202496 685 0 0 0 5 0 0 0 18 0 1 0 1132665695 3256320 641 4294967295 134512640 135237372 3218452640 3218452088 134732614 0 0 0 0 0 0 0 17 1 0 0 0
/proc/16742/statm: 795 641 75 177 0 617 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 3180

[startup+0.111037 s]
/proc/loadavg: 4.09 4.10 3.74 3/99 16742
/proc/meminfo: memFree=3046168/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=5756 CPUtime=0.11
/proc/16742/stat : 16742 (ttsth-5-0) R 16740 16742 20984 0 -1 4202496 1317 0 0 0 11 0 0 0 18 0 1 0 1132665695 5894144 1273 4294967295 134512640 135237372 3218452640 3218451164 134639534 0 0 0 0 0 0 0 17 1 0 0 0
/proc/16742/statm: 1439 1274 75 177 0 1261 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 5756

[startup+0.311019 s]
/proc/loadavg: 4.09 4.10 3.74 3/99 16742
/proc/meminfo: memFree=3046168/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=11084 CPUtime=0.31
/proc/16742/stat : 16742 (ttsth-5-0) R 16740 16742 20984 0 -1 4202496 2561 0 0 0 31 0 0 0 18 0 1 0 1132665695 11350016 2517 4294967295 134512640 135237372 3218452640 3218452332 134534043 0 0 0 0 0 0 0 17 1 0 0 0
/proc/16742/statm: 2771 2517 75 177 0 2593 0
Current children cumulated CPU time (s) 0.31
Current children cumulated vsize (KiB) 11084

[startup+0.710982 s]
/proc/loadavg: 4.09 4.10 3.74 3/99 16742
/proc/meminfo: memFree=3046168/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=21972 CPUtime=0.71
/proc/16742/stat : 16742 (ttsth-5-0) R 16740 16742 20984 0 -1 4202496 5145 0 0 0 70 1 0 0 18 0 1 0 1132665695 22499328 5100 4294967295 134512640 135237372 3218452640 3218452144 134540092 0 0 0 0 0 0 0 17 1 0 0 0
/proc/16742/statm: 5493 5100 76 177 0 5315 0
Current children cumulated CPU time (s) 0.71
Current children cumulated vsize (KiB) 21972

[startup+1.5109 s]
/proc/loadavg: 4.09 4.10 3.74 2/100 16743
/proc/meminfo: memFree=3024264/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=24724 CPUtime=1.51
/proc/16742/stat : 16742 (ttsth-5-0) R 16740 16742 20984 0 -1 4202496 6413 0 0 0 150 1 0 0 20 0 1 0 1132665695 25317376 5982 4294967295 134512640 135237372 3218452640 3218451968 134555272 0 0 0 0 0 0 0 17 1 0 0 0
/proc/16742/statm: 6181 5982 77 177 0 6003 0
Current children cumulated CPU time (s) 1.51
Current children cumulated vsize (KiB) 24724

[startup+3.11075 s]
/proc/loadavg: 4.09 4.10 3.74 2/100 16743
/proc/meminfo: memFree=3022548/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=24860 CPUtime=3.11
/proc/16742/stat : 16742 (ttsth-5-0) R 16740 16742 20984 0 -1 4202496 6443 0 0 0 310 1 0 0 25 0 1 0 1132665695 25456640 6012 4294967295 134512640 135237372 3218452640 3218452000 134600733 0 0 0 0 0 0 0 17 1 0 0 0
/proc/16742/statm: 6215 6012 77 177 0 6037 0
Current children cumulated CPU time (s) 3.11
Current children cumulated vsize (KiB) 24860

[startup+6.31045 s]
/proc/loadavg: 3.84 4.05 3.73 2/100 16743
/proc/meminfo: memFree=3022332/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=25124 CPUtime=6.31
/proc/16742/stat : 16742 (ttsth-5-0) R 16740 16742 20984 0 -1 4202496 6486 0 0 0 630 1 0 0 25 0 1 0 1132665695 25726976 6055 4294967295 134512640 135237372 3218452640 3218452000 134600718 0 0 0 0 0 0 0 17 1 0 0 0
/proc/16742/statm: 6281 6055 77 177 0 6103 0
Current children cumulated CPU time (s) 6.31
Current children cumulated vsize (KiB) 25124

[startup+12.7098 s]
/proc/loadavg: 3.62 4.00 3.71 5/104 16747
/proc/meminfo: memFree=2919220/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=166112 CPUtime=20.4
/proc/16742/stat : 16742 (ttsth-5-0) S 16740 16742 20984 0 -1 4202496 34263 0 0 0 2029 11 0 0 25 0 5 0 1132665695 170098688 33578 4294967295 134512640 135237372 3218452640 3218452236 134629544 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 41528 33578 95 177 0 41350 0
[pid=16742/tid=16744] ppid=16740 vsize=166112 CPUtime=2.58
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 1270 0 0 0 258 0 0 0 25 0 5 0 1132666706 170098688 33578 4294967295 134512640 135237372 3218452640 3081989744 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=16742/tid=16745] ppid=16740 vsize=166112 CPUtime=2.56
/proc/16742/task/16745/stat : 16745 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 1385 0 0 0 254 2 0 0 25 0 5 0 1132666706 170098688 33578 4294967295 134512640 135237372 3218452640 3073610416 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=16742/tid=16746] ppid=16740 vsize=166112 CPUtime=2.58
/proc/16742/task/16746/stat : 16746 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 1644 0 0 0 258 0 0 0 25 0 5 0 1132666706 170098688 33578 4294967295 134512640 135237372 3218452640 3065307968 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=16742/tid=16747] ppid=16740 vsize=166112 CPUtime=2.56
/proc/16742/task/16747/stat : 16747 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 1229 0 0 0 255 1 0 0 25 0 5 0 1132666706 170098688 33578 4294967295 134512640 135237372 3218452640 3056823120 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 20.4
Current children cumulated vsize (KiB) 166112

[startup+25.5086 s]
/proc/loadavg: 3.70 4.00 3.72 5/104 16750
/proc/meminfo: memFree=2836920/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=247616 CPUtime=71.6
/proc/16742/stat : 16742 (ttsth-5-0) S 16740 16742 20984 0 -1 4202496 53825 0 0 0 7141 19 0 0 25 0 5 0 1132665695 253558784 53140 4294967295 134512640 135237372 3218452640 3218452236 134629544 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 61904 53140 95 177 0 61726 0
[pid=16742/tid=16744] ppid=16740 vsize=247616 CPUtime=15.38
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 3964 0 0 0 1536 2 0 0 25 0 5 0 1132666706 253558784 53140 4294967295 134512640 135237372 3218452640 3082124336 134749353 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=16742/tid=16745] ppid=16740 vsize=247616 CPUtime=15.36
/proc/16742/task/16745/stat : 16745 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 5516 0 0 0 1533 3 0 0 25 0 5 0 1132666706 253558784 53140 4294967295 134512640 135237372 3218452640 3073667048 134732535 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=16742/tid=16746] ppid=16740 vsize=247616 CPUtime=15.38
/proc/16742/task/16746/stat : 16746 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 8054 0 0 0 1535 3 0 0 25 0 5 0 1132666706 253558784 53140 4294967295 134512640 135237372 3218452640 3065311672 134519962 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=16742/tid=16747] ppid=16740 vsize=247616 CPUtime=15.36
/proc/16742/task/16747/stat : 16747 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 7556 0 0 0 1533 3 0 0 25 0 5 0 1132666706 253558784 53140 4294967295 134512640 135237372 3218452640 3056832624 134546034 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 71.6
Current children cumulated vsize (KiB) 247616

[startup+51.1062 s]
/proc/loadavg: 3.87 4.02 3.73 5/104 16934
/proc/meminfo: memFree=2690952/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=402112 CPUtime=173.37
/proc/16742/stat : 16742 (ttsth-5-0) S 16740 16742 20984 0 -1 4202496 90579 0 0 0 17300 37 0 0 25 0 5 0 1132665695 411762688 89894 4294967295 134512640 135237372 3218452640 3218452236 134629544 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 100528 89894 95 177 0 100350 0
[pid=16742/tid=16744] ppid=16740 vsize=402112 CPUtime=40.94
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 12898 0 0 0 4089 5 0 0 25 0 5 0 1132666706 411762688 89894 4294967295 134512640 135237372 3218452640 3082040400 134588538 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=16742/tid=16745] ppid=16740 vsize=402112 CPUtime=40.85
/proc/16742/task/16745/stat : 16745 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 14751 0 0 0 4078 7 0 0 25 0 5 0 1132666706 411762688 89894 4294967295 134512640 135237372 3218452640 3073740460 134636641 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=16742/tid=16746] ppid=16740 vsize=402112 CPUtime=40.55
/proc/16742/task/16746/stat : 16746 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 16239 0 0 0 4047 8 0 0 25 0 5 0 1132666706 411762688 89894 4294967295 134512640 135237372 3218452640 3065342528 134546026 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=16742/tid=16747] ppid=16740 vsize=402112 CPUtime=40.91
/proc/16742/task/16747/stat : 16747 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 17956 0 0 0 4082 9 0 0 25 0 5 0 1132666706 411762688 89894 4294967295 134512640 135237372 3218452640 3056818416 134546034 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 173.37
Current children cumulated vsize (KiB) 402112

[startup+102.311 s]
/proc/loadavg: 4.06 4.04 3.75 6/105 17007
/proc/meminfo: memFree=2502728/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=599204 CPUtime=378.17
/proc/16742/stat : 16742 (ttsth-5-0) S 16740 16742 20984 0 -1 4202496 137025 0 0 0 37754 63 0 0 25 0 5 0 1132665695 613584896 136340 4294967295 134512640 135237372 3218452640 3218452236 134629544 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 149801 136340 95 177 0 149623 0
[pid=16742/tid=16744] ppid=16740 vsize=599204 CPUtime=92.15
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 25226 0 0 0 9205 10 0 0 25 0 5 0 1132666706 613584896 136340 4294967295 134512640 135237372 3218452640 3082130080 134552748 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=16742/tid=16745] ppid=16740 vsize=599204 CPUtime=92.05
/proc/16742/task/16745/stat : 16745 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 28115 0 0 0 9189 16 0 0 25 0 5 0 1132666706 613584896 136340 4294967295 134512640 135237372 3218452640 3073697632 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=16742/tid=16746] ppid=16740 vsize=599204 CPUtime=91.75
/proc/16742/task/16746/stat : 16746 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 25202 0 0 0 9161 14 0 0 25 0 5 0 1132666706 613584896 136340 4294967295 134512640 135237372 3218452640 3065342048 134546026 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=16742/tid=16747] ppid=16740 vsize=599204 CPUtime=92.1
/proc/16742/task/16747/stat : 16747 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 29747 0 0 0 9195 15 0 0 25 0 5 0 1132666706 613584896 136340 4294967295 134512640 135237372 3218452640 3056920944 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 378.17
Current children cumulated vsize (KiB) 599204

[startup+162.306 s]
/proc/loadavg: 4.47 4.17 3.81 5/104 17310
/proc/meminfo: memFree=2313368/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=799260 CPUtime=617.81
/proc/16742/stat : 16742 (ttsth-5-0) S 16740 16742 20984 0 -1 4202496 184692 0 0 0 61692 89 0 0 25 0 5 0 1132665695 818442240 184007 4294967295 134512640 135237372 3218452640 3218452236 134629544 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 199815 184007 95 177 0 199637 0
[pid=16742/tid=16744] ppid=16740 vsize=799260 CPUtime=152.14
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 38482 0 0 0 15201 13 0 0 25 0 5 0 1132666706 818442240 184007 4294967295 134512640 135237372 3218452640 3082068928 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=16742/tid=16745] ppid=16740 vsize=799260 CPUtime=151.98
/proc/16742/task/16745/stat : 16745 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 40870 0 0 0 15177 21 0 0 25 0 5 0 1132666706 818442240 184007 4294967295 134512640 135237372 3218452640 3073695616 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=16742/tid=16746] ppid=16740 vsize=799260 CPUtime=151.58
/proc/16742/task/16746/stat : 16746 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 34039 0 0 0 15140 18 0 0 25 0 5 0 1132666706 818442240 184007 4294967295 134512640 135237372 3218452640 3065330684 134636641 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=16742/tid=16747] ppid=16740 vsize=799260 CPUtime=151.99
/proc/16742/task/16747/stat : 16747 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 42566 0 0 0 15170 29 0 0 25 0 5 0 1132666706 818442240 184007 4294967295 134512640 135237372 3218452640 3056915376 134546026 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 617.81
Current children cumulated vsize (KiB) 799260

[startup+222.31 s]
/proc/loadavg: 4.33 4.18 3.83 7/105 17360
/proc/meminfo: memFree=2126576/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=994992 CPUtime=857.76
/proc/16742/stat : 16742 (ttsth-5-0) S 16740 16742 20984 0 -1 4202496 230935 0 0 0 85660 116 0 0 25 0 5 0 1132665695 1018871808 230250 4294967295 134512640 135237372 3218452640 3218452236 134629544 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 248748 230250 95 177 0 248570 0
[pid=16742/tid=16744] ppid=16740 vsize=994992 CPUtime=212.15
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 47026 0 0 0 21196 19 0 0 25 0 5 0 1132666706 1018871808 230250 4294967295 134512640 135237372 3218452640 3081996304 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=16742/tid=16745] ppid=16740 vsize=994992 CPUtime=211.92
/proc/16742/task/16745/stat : 16745 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 59091 0 0 0 21160 32 0 0 25 0 5 0 1132666706 1018871808 230250 4294967295 134512640 135237372 3218452640 3073701580 134636641 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=16742/tid=16746] ppid=16740 vsize=994992 CPUtime=211.59
/proc/16742/task/16746/stat : 16746 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 42468 0 0 0 21138 21 0 0 25 0 5 0 1132666706 1018871808 230250 4294967295 134512640 135237372 3218452640 3065313200 134546026 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=16742/tid=16747] ppid=16740 vsize=994992 CPUtime=211.98
/proc/16742/task/16747/stat : 16747 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 53615 0 0 0 21162 36 0 0 25 0 5 0 1132666706 1018871808 230250 4294967295 134512640 135237372 3218452640 3056895852 134636641 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 857.76
Current children cumulated vsize (KiB) 994992

[startup+282.314 s]
/proc/loadavg: 4.53 4.28 3.89 6/105 17539
/proc/meminfo: memFree=1913676/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=1226120 CPUtime=1097.4
/proc/16742/stat : 16742 (ttsth-5-0) S 16740 16742 20984 0 -1 4202496 285112 0 0 0 109594 146 0 0 25 0 5 0 1132665695 1255546880 284427 4294967295 134512640 135237372 3218452640 3218452236 134629544 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 306530 284427 95 177 0 306352 0
[pid=16742/tid=16744] ppid=16740 vsize=1226120 CPUtime=272.14
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 61312 0 0 0 27190 24 0 0 25 0 5 0 1132666706 1255546880 284427 4294967295 134512640 135237372 3218452640 3082082864 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=16742/tid=16745] ppid=16740 vsize=1226120 CPUtime=271.88
/proc/16742/task/16745/stat : 16745 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 72697 0 0 0 27147 41 0 0 25 0 5 0 1132666706 1255546880 284427 4294967295 134512640 135237372 3218452640 3073704028 134636641 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=16742/tid=16746] ppid=16740 vsize=1226120 CPUtime=271.4
/proc/16742/task/16746/stat : 16746 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 57984 0 0 0 27109 31 0 0 25 0 5 0 1132666706 1255546880 284427 4294967295 134512640 135237372 3218452640 3065298284 134636641 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=16742/tid=16747] ppid=16740 vsize=1226120 CPUtime=271.86
/proc/16742/task/16747/stat : 16747 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 64384 0 0 0 27144 42 0 0 25 0 5 0 1132666706 1255546880 284427 4294967295 134512640 135237372 3218452640 3056948412 134636641 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1097.4
Current children cumulated vsize (KiB) 1226120

[startup+342.308 s]
/proc/loadavg: 4.36 4.28 3.91 5/104 17879
/proc/meminfo: memFree=1666444/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=1482880 CPUtime=1336.71
/proc/16742/stat : 16742 (ttsth-5-0) S 16740 16742 20984 0 -1 4202496 345652 0 0 0 133479 192 0 0 25 0 5 0 1132665695 1518469120 344967 4294967295 134512640 135237372 3218452640 3218452236 134629544 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 370720 344967 95 177 0 370542 0
[pid=16742/tid=16744] ppid=16740 vsize=1482880 CPUtime=332.11
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 79526 0 0 0 33175 36 0 0 25 0 5 0 1132666706 1518469120 344967 4294967295 134512640 135237372 3218452640 3082128336 134562328 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=16742/tid=16745] ppid=16740 vsize=1482880 CPUtime=331.8
/proc/16742/task/16745/stat : 16745 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 84264 0 0 0 33129 51 0 0 25 0 5 0 1132666706 1518469120 344967 4294967295 134512640 135237372 3218452640 3073612048 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=16742/tid=16746] ppid=16740 vsize=1482880 CPUtime=331.12
/proc/16742/task/16746/stat : 16746 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 75727 0 0 0 33063 49 0 0 25 0 5 0 1132666706 1518469120 344967 4294967295 134512640 135237372 3218452640 3065214704 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=16742/tid=16747] ppid=16740 vsize=1482880 CPUtime=331.56
/proc/16742/task/16747/stat : 16747 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 77400 0 0 0 33108 48 0 0 25 0 5 0 1132666706 1518469120 344967 4294967295 134512640 135237372 3218452640 3056919792 134680503 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1336.71
Current children cumulated vsize (KiB) 1482880

[startup+402.303 s]
/proc/loadavg: 4.16 4.24 3.92 6/105 17900
/proc/meminfo: memFree=1486244/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=1677224 CPUtime=1576.7
/proc/16742/stat : 16742 (ttsth-5-0) S 16740 16742 20984 0 -1 4202496 391309 0 0 0 157450 220 0 0 25 0 5 0 1132665695 1717477376 390624 4294967295 134512640 135237372 3218452640 3218452236 134629544 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 419306 390625 95 177 0 419128 0
[pid=16742/tid=16744] ppid=16740 vsize=1677224 CPUtime=392.11
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 91007 0 0 0 39172 39 0 0 25 0 5 0 1132666706 1717477376 390625 4294967295 134512640 135237372 3218452640 3081987632 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=16742/tid=16745] ppid=16740 vsize=1677224 CPUtime=391.8
/proc/16742/task/16745/stat : 16745 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 97575 0 0 0 39121 59 0 0 25 0 5 0 1132666706 1717477376 390625 4294967295 134512640 135237372 3218452640 3073697296 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=16742/tid=16746] ppid=16740 vsize=1677224 CPUtime=391.11
/proc/16742/task/16746/stat : 16746 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 86700 0 0 0 39054 57 0 0 25 0 5 0 1132666706 1717477376 390625 4294967295 134512640 135237372 3218452640 3065342720 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=16742/tid=16747] ppid=16740 vsize=1677224 CPUtime=391.56
/proc/16742/task/16747/stat : 16747 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 87293 0 0 0 39099 57 0 0 25 0 5 0 1132666706 1717477376 390625 4294967295 134512640 135237372 3218452640 3056864544 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1576.7
Current children cumulated vsize (KiB) 1677224

[startup+462.307 s]
/proc/loadavg: 4.15 4.22 3.93 6/105 18089
/proc/meminfo: memFree=1244620/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=1931828 CPUtime=1816.33
/proc/16742/stat : 16742 (ttsth-5-0) S 16740 16742 20984 0 -1 4202496 451332 0 0 0 181378 255 0 0 25 0 5 0 1132665695 1978191872 450647 4294967295 134512640 135237372 3218452640 3218452236 134629544 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 482957 450647 95 177 0 482779 0
[pid=16742/tid=16744] ppid=16740 vsize=1931828 CPUtime=452.12
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 107062 0 0 0 45164 48 0 0 25 0 5 0 1132666706 1978191872 450647 4294967295 134512640 135237372 3218452640 3082107160 134519944 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=16742/tid=16745] ppid=16740 vsize=1931828 CPUtime=451.72
/proc/16742/task/16745/stat : 16745 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 112569 0 0 0 45105 67 0 0 25 0 5 0 1132666706 1978191872 450647 4294967295 134512640 135237372 3218452640 3073697548 134636641 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=16742/tid=16746] ppid=16740 vsize=1931828 CPUtime=451.03
/proc/16742/task/16746/stat : 16746 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 102743 0 0 0 45038 65 0 0 25 0 5 0 1132666706 1978191872 450647 4294967295 134512640 135237372 3218452640 3065306864 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=16742/tid=16747] ppid=16740 vsize=1931828 CPUtime=451.34
/proc/16742/task/16747/stat : 16747 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 100223 0 0 0 45067 67 0 0 25 0 5 0 1132666706 1978191872 450647 4294967295 134512640 135237372 3218452640 3056953776 134546026 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1816.33
Current children cumulated vsize (KiB) 1931828



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+502.713 s]
/proc/loadavg: 4.38 4.28 3.96 5/104 18257
/proc/meminfo: memFree=1087608/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=2096188 CPUtime=1977.75
/proc/16742/stat : 16742 (ttsth-5-0) S 16740 16742 20984 0 -1 4202496 490172 0 0 0 197497 278 0 0 25 0 5 0 1132665695 2146496512 489487 4294967295 134512640 135237372 3218452640 3218452236 134629544 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 524047 489487 95 177 0 523869 0
[pid=16742/tid=16744] ppid=16740 vsize=2096188 CPUtime=492.49
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 118090 0 0 0 49197 52 0 0 25 0 5 0 1132666706 2146496512 489487 4294967295 134512640 135237372 3218452640 3082031184 134587921 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=16742/tid=16745] ppid=16740 vsize=2096188 CPUtime=492.13
/proc/16742/task/16745/stat : 16745 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 122127 0 0 0 49142 71 0 0 25 0 5 0 1132666706 2146496512 489487 4294967295 134512640 135237372 3218452640 3073705888 134680467 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=16742/tid=16746] ppid=16740 vsize=2096188 CPUtime=491.35
/proc/16742/task/16746/stat : 16746 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 111139 0 0 0 49064 71 0 0 25 0 5 0 1132666706 2146496512 489487 4294967295 134512640 135237372 3218452640 3065305664 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=16742/tid=16747] ppid=16740 vsize=2096188 CPUtime=491.66
/proc/16742/task/16747/stat : 16747 (ttsth-5-0) R 16740 16742 20984 0 -1 4202560 110081 0 0 0 49090 76 0 0 25 0 5 0 1132666706 2146496512 489487 4294967295 134512640 135237372 3218452640 3056913312 134546034 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1977.75
Current children cumulated vsize (KiB) 2096188

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+502.833 s]
/proc/loadavg: 4.38 4.28 3.96 5/104 18257
/proc/meminfo: memFree=1087608/3368532 swapFree=1953580/1959888
[pid=16742] ppid=16740 vsize=0 CPUtime=1977.9
/proc/16742/stat : 16742 (ttsth-5-0) Z 16740 16742 20984 0 -1 4203524 490197 0 0 0 197502 288 0 0 18 0 2 0 1132665695 0 0 4294967295 0 0 0 0 0 0 0 0 0 4294967295 0 0 17 1 0 0 0
/proc/16742/statm: 0 0 0 0 0 0 0
[pid=16742/tid=16744] ppid=16740 vsize=0 CPUtime=492.61
/proc/16742/task/16744/stat : 16744 (ttsth-5-0) R 16740 16742 20984 0 -1 4203588 118094 0 0 0 49199 62 0 0 25 0 2 0 1132666706 0 0 4294967295 0 0 0 0 0 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1977.9
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 16742 and gives
#  childrusage.ru_utime.tv_sec=1975
#  childrusage.ru_utime.tv_usec=20000
#  childrusage.ru_stime.tv_sec=2
#  childrusage.ru_stime.tv_usec=880000
# CPU time returned by wait4() is 1977.9
# while last known CPU time is 1977.9
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 502.838
CPU time (s): 1977.9
CPU user time (s): 1975.02
CPU system time (s): 2.88
CPU usage (%): 393.347
Max. virtual memory (cumulated for all children) (KiB): 2096188

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1975.02
system time used= 2.88
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 490197
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3089
involuntary context switches= 35941

runsolver used 0 second user time and 0.01 second system time

The end

Launcher Data 

Begin job on  at 2009-05-30 12:40:13
IDJOB=1779973
IDBENCH=70944
IDSOLVER=632
FILE ID=/1779973-1243680013
PBS_JOBID=
Free space on /tmp= 15883 MiB

SOLVER NAME= ttsth-5-0 2009-03-19 (4 threads)
BENCH NAME= SAT09/APPLICATIONS/bioinfo/q_query_3_l41_lambda.cnf
COMMAND LINE= HOME/ttsth-5-0 -num-threads MAXNBTHREAD BENCHNAME
RUNSOLVER COMMAND LINE= runsolver --timestamp  -w /tmp/evaluation-result-1779973-1243680013/watcher-1779973-1243680013 -o /tmp/evaluation-result-1779973-1243680013/solver-1779973-1243680013 -C 10000 -W 1000 -M 2047  HOME/ttsth-5-0 -num-threads 4 HOME/instance-1779973-1243680013.cnf

TIME LIMIT= 10000 seconds
MEMORY LIMIT= 2047 MiB
MAX NB THREAD= 4

MD5SUM BENCH= 50760ceab82aaa514e253121456f71f6
RANDOM SEED= 509644346

Linux (none) 2.6.21.5 #6 SMP Wed Jul 4 17:16:38 CEST 2007 i686 GNU/Linux

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.105
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3992.42
clflush size	: 64

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.105
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.11
clflush size	: 64

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.105
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.16
clflush size	: 64

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.105
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.15
clflush size	: 64


/proc/meminfo:
MemTotal:      3368532 kB
MemFree:       3046612 kB
Buffers:         87368 kB
Cached:         130520 kB
SwapCached:       2424 kB
Active:         144412 kB
Inactive:        83800 kB
HighTotal:     2489216 kB
HighFree:      2345816 kB
LowTotal:       879316 kB
LowFree:        700796 kB
SwapTotal:     1959888 kB
SwapFree:      1953580 kB
Dirty:            3572 kB
Writeback:           0 kB
AnonPages:        9408 kB
Mapped:           4764 kB
Slab:            84176 kB
SReclaimable:    75000 kB
SUnreclaim:       9176 kB
PageTables:        508 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   3644152 kB
Committed_AS:    93568 kB
VmallocTotal:   114680 kB
VmallocUsed:      1444 kB
VmallocChunk:   113072 kB

Free space on /tmp at the end= 15883 MiB
End job on  at 2009-05-30 12:48:38