Trace number 1529718

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
HydraSAT-Multi 2009-03-22-Multi? (TO) 1200.03 1200.71

General information on the benchmark

NameRANDOM/LARGE/3SAT/UNKNOWN/18000/
unif-k3-r4.2-v18000-c75600-S2088061175-062.cnf
MD5SUM0b73ddee1057244eb6a8d3ac4fd7781f
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark136.921
Satisfiable
(Un)Satisfiability was proved
Number of variables18000
Number of clauses75600
Sum of the clauses size226800
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 375600
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c Trying configuration 1
700.50/700.98	HOME/hydrasat-multi.sh: line 48:  3299 CPU time limit exceeded (core dumped) $SOLVER1 $BENCHNAME $OPTIONS1
700.50/700.98	c Trying configuration 2

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver version 3.2.9 (svn:489) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-1529718-1238405816/watcher-1529718-1238405816 -o /tmp/evaluation-result-1529718-1238405816/solver-1529718-1238405816 -C 1200 -W 1800 -M 1800 --output-limit 1,15 HOME/hydrasat-multi.sh HOME HOME/instance-1529718-1238405816.cnf 1800 105379673 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1200 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1230 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 1843200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 1894400 KiB
Solver output will be limited to a maximum of 15728640 bytes. The first 1048576 bytes and the last 14680064 bytes will be preserved
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.92 0.99 1.13 3/72 3298
/proc/meminfo: memFree=1108984/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=1748 CPUtime=0
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 175 0 0 0 0 0 0 0 19 0 1 0 1113242817 1789952 157 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 437 157 117 317 0 117 0

[startup+0.0886709 s]
/proc/loadavg: 0.92 0.99 1.13 3/72 3298
/proc/meminfo: memFree=1108984/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=5048 CPUtime=0.07
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 1042 0 0 0 7 0 0 0 19 0 1 0 1113242817 5169152 1024 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 1262 1024 117 317 0 942 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 10408

[startup+0.101672 s]
/proc/loadavg: 0.92 0.99 1.13 3/72 3298
/proc/meminfo: memFree=1108984/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=5576 CPUtime=0.09
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 1177 0 0 0 9 0 0 0 19 0 1 0 1113242817 5709824 1159 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 1394 1159 117 317 0 1074 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 10936

[startup+0.301711 s]
/proc/loadavg: 0.92 0.99 1.13 3/72 3298
/proc/meminfo: memFree=1108984/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=9080 CPUtime=0.29
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 2104 0 0 0 28 1 0 0 21 0 1 0 1113242817 9297920 1905 1992294400 134512640 135813592 4294956160 18446744073709551615 134687986 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 2270 1905 126 317 0 1950 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 14440

[startup+0.70179 s]
/proc/loadavg: 0.92 0.99 1.13 3/72 3298
/proc/meminfo: memFree=1108984/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=13644 CPUtime=0.69
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 3389 0 0 0 68 1 0 0 24 0 1 0 1113242817 13971456 3083 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 3411 3083 150 317 0 3091 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 19004

[startup+1.50195 s]
/proc/loadavg: 0.92 0.99 1.13 2/74 3300
/proc/meminfo: memFree=1096296/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=14436 CPUtime=1.48
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 3594 0 0 0 147 1 0 0 25 0 1 0 1113242817 14782464 3288 1992294400 134512640 135813592 4294956160 18446744073709551615 134664877 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 3609 3288 150 317 0 3289 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 19796

[startup+3.10126 s]
/proc/loadavg: 0.92 0.99 1.13 2/74 3300
/proc/meminfo: memFree=1095464/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=15492 CPUtime=3.09
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 3838 0 0 0 307 2 0 0 25 0 1 0 1113242817 15863808 3532 1992294400 134512640 135813592 4294956160 18446744073709551615 134609004 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 3873 3532 150 317 0 3553 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 20852

[startup+6.30189 s]
/proc/loadavg: 0.93 0.99 1.12 2/74 3300
/proc/meminfo: memFree=1093224/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=17096 CPUtime=6.29
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 4239 0 0 0 627 2 0 0 25 0 1 0 1113242817 17506304 3933 1992294400 134512640 135813592 4294956160 18446744073709551615 134665557 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 4274 3933 150 317 0 3954 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 22456

[startup+12.7012 s]
/proc/loadavg: 0.93 0.99 1.12 2/74 3300
/proc/meminfo: memFree=1090152/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=20152 CPUtime=12.69
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 5003 0 0 0 1266 3 0 0 25 0 1 0 1113242817 20635648 4697 1992294400 134512640 135813592 4294956160 18446744073709551615 134665314 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 5038 4697 150 317 0 4718 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 25512

[startup+25.5017 s]
/proc/loadavg: 0.95 0.99 1.12 2/74 3300
/proc/meminfo: memFree=1083752/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=26964 CPUtime=25.47
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 6708 0 0 0 2543 4 0 0 25 0 1 0 1113242817 27611136 6402 1992294400 134512640 135813592 4294956160 18446744073709551615 134664998 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 6741 6402 150 317 0 6421 0
Current children cumulated CPU time (s) 25.47
Current children cumulated vsize (KiB) 32324

[startup+51.1017 s]
/proc/loadavg: 0.96 0.99 1.12 2/74 3300
/proc/meminfo: memFree=1075496/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=34872 CPUtime=51.05
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 8685 0 0 0 5100 5 0 0 25 0 1 0 1113242817 35708928 8379 1992294400 134512640 135813592 4294956160 18446744073709551615 134665314 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 8718 8379 150 317 0 8398 0
Current children cumulated CPU time (s) 51.05
Current children cumulated vsize (KiB) 40232

[startup+102.307 s]
/proc/loadavg: 0.98 0.99 1.11 2/74 3300
/proc/meminfo: memFree=1061864/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=48520 CPUtime=102.23
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 12090 0 0 0 10216 7 0 0 25 0 1 0 1113242817 49684480 11784 1992294400 134512640 135813592 4294956160 18446744073709551615 134665442 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 12130 11784 150 317 0 11810 0
Current children cumulated CPU time (s) 102.23
Current children cumulated vsize (KiB) 53880

[startup+162.303 s]
/proc/loadavg: 0.99 0.99 1.09 2/74 3300
/proc/meminfo: memFree=1048552/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=62160 CPUtime=162.19
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 15502 0 0 0 16209 10 0 0 25 0 1 0 1113242817 63651840 15196 1992294400 134512640 135813592 4294956160 18446744073709551615 134665297 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 15540 15196 150 317 0 15220 0
Current children cumulated CPU time (s) 162.19
Current children cumulated vsize (KiB) 67520

[startup+222.301 s]
/proc/loadavg: 0.99 0.99 1.09 2/74 3300
/proc/meminfo: memFree=1011816/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=98800 CPUtime=222.16
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 24563 0 0 0 22198 18 0 0 25 0 1 0 1113242817 101171200 24257 1992294400 134512640 135813592 4294956160 18446744073709551615 134665314 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 24700 24257 150 317 0 24380 0
Current children cumulated CPU time (s) 222.16
Current children cumulated vsize (KiB) 104160

[startup+282.301 s]
/proc/loadavg: 0.99 0.99 1.08 2/74 3300
/proc/meminfo: memFree=1011816/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=98800 CPUtime=282.13
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 24563 0 0 0 28195 18 0 0 25 0 1 0 1113242817 101171200 24257 1992294400 134512640 135813592 4294956160 18446744073709551615 134665314 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 24700 24257 150 317 0 24380 0
Current children cumulated CPU time (s) 282.13
Current children cumulated vsize (KiB) 104160

[startup+342.302 s]
/proc/loadavg: 0.99 0.99 1.07 2/74 3300
/proc/meminfo: memFree=998184/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=112956 CPUtime=342.09
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 28016 0 0 0 34188 21 0 0 25 0 1 0 1113242817 115666944 27710 1992294400 134512640 135813592 4294956160 18446744073709551615 134665511 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 28239 27710 150 317 0 27919 0

################
# More data... #
################

/proc/3299/statm: 35162 34560 150 317 0 34842 0
Current children cumulated CPU time (s) 581.96
Current children cumulated vsize (KiB) 146008

[startup+642.301 s]
/proc/loadavg: 0.99 0.99 1.05 2/74 3302
/proc/meminfo: memFree=950632/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5360 CPUtime=0
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 1113242817 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1340 239 197 169 0 51 0
[pid=3299] ppid=3298 vsize=160772 CPUtime=641.94
/proc/3299/stat : 3299 (hydrasat_cache4) R 3298 3298 2979 0 -1 4194304 39853 0 0 0 64163 31 0 0 25 0 1 0 1113242817 164630528 39547 1992294400 134512640 135813592 4294956160 18446744073709551615 134665442 0 0 4096 1 0 0 0 17 0 0 0
/proc/3299/statm: 40193 39547 150 317 0 39873 0
Current children cumulated CPU time (s) 641.94
Current children cumulated vsize (KiB) 166132

[startup+702.302 s]
/proc/loadavg: 0.99 0.99 1.04 2/74 3303
/proc/meminfo: memFree=919848/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5364 CPUtime=700.58
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 321 45096 0 0 0 0 69955 103 19 0 1 0 1113242817 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1341 245 202 169 0 52 0
[pid=3303] ppid=3298 vsize=9812 CPUtime=1.31
/proc/3303/stat : 3303 (hydrasat_cachew) R 3298 3298 2979 0 -1 4194304 2411 0 0 0 130 1 0 0 25 0 1 0 1113312915 10047488 2106 1992294400 134512640 135995732 4294956016 18446744073709551615 134807701 0 0 4096 1 0 0 0 17 0 0 0
/proc/3303/statm: 2453 2106 172 362 0 2088 0
Current children cumulated CPU time (s) 701.89
Current children cumulated vsize (KiB) 15176

[startup+762.306 s]
/proc/loadavg: 1.07 1.01 1.04 2/74 3303
/proc/meminfo: memFree=901864/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5364 CPUtime=700.58
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 321 45096 0 0 0 0 69955 103 19 0 1 0 1113242817 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1341 245 202 169 0 52 0
[pid=3303] ppid=3298 vsize=31340 CPUtime=61.28
/proc/3303/stat : 3303 (hydrasat_cachew) R 3298 3298 2979 0 -1 4194304 7777 0 0 0 6122 6 0 0 25 0 1 0 1113312915 32092160 7472 1992294400 134512640 135995732 4294956016 18446744073709551615 134807701 0 0 4096 1 0 0 0 17 0 0 0
/proc/3303/statm: 7835 7472 172 362 0 7470 0
Current children cumulated CPU time (s) 761.86
Current children cumulated vsize (KiB) 36704

[startup+822.302 s]
/proc/loadavg: 1.02 1.01 1.04 2/74 3303
/proc/meminfo: memFree=880936/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5364 CPUtime=700.58
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 321 45096 0 0 0 0 69955 103 19 0 1 0 1113242817 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1341 245 202 169 0 52 0
[pid=3303] ppid=3298 vsize=52320 CPUtime=121.24
/proc/3303/stat : 3303 (hydrasat_cachew) R 3298 3298 2979 0 -1 4194304 13022 0 0 0 12114 10 0 0 25 0 1 0 1113312915 53575680 12717 1992294400 134512640 135995732 4294956016 18446744073709551615 134807904 0 0 4096 1 0 0 0 17 0 0 0
/proc/3303/statm: 13080 12717 172 362 0 12715 0
Current children cumulated CPU time (s) 821.82
Current children cumulated vsize (KiB) 57684

[startup+882.301 s]
/proc/loadavg: 1.01 1.00 1.03 2/74 3303
/proc/meminfo: memFree=870568/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5364 CPUtime=700.58
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 321 45096 0 0 0 0 69955 103 19 0 1 0 1113242817 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1341 245 202 169 0 52 0
[pid=3303] ppid=3298 vsize=62904 CPUtime=181.21
/proc/3303/stat : 3303 (hydrasat_cachew) R 3298 3298 2979 0 -1 4194304 15676 0 0 0 18109 12 0 0 25 0 1 0 1113312915 64413696 15371 1992294400 134512640 135995732 4294956016 18446744073709551615 134807747 0 0 4096 1 0 0 0 17 0 0 0
/proc/3303/statm: 15726 15371 172 362 0 15361 0
Current children cumulated CPU time (s) 881.79
Current children cumulated vsize (KiB) 68268

[startup+942.301 s]
/proc/loadavg: 1.00 1.00 1.03 2/74 3303
/proc/meminfo: memFree=856872/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5364 CPUtime=700.58
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 321 45096 0 0 0 0 69955 103 19 0 1 0 1113242817 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1341 245 202 169 0 52 0
[pid=3303] ppid=3298 vsize=76380 CPUtime=241.18
/proc/3303/stat : 3303 (hydrasat_cachew) R 3298 3298 2979 0 -1 4194304 19041 0 0 0 24103 15 0 0 25 0 1 0 1113312915 78213120 18736 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 0 0 0
/proc/3303/statm: 19095 18736 172 362 0 18730 0
Current children cumulated CPU time (s) 941.76
Current children cumulated vsize (KiB) 81744

[startup+1002.3 s]
/proc/loadavg: 1.00 1.00 1.02 2/74 3303
/proc/meminfo: memFree=841192/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5364 CPUtime=700.58
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 321 45096 0 0 0 0 69955 103 19 0 1 0 1113242817 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1341 245 202 169 0 52 0
[pid=3303] ppid=3298 vsize=92256 CPUtime=301.15
/proc/3303/stat : 3303 (hydrasat_cachew) R 3298 3298 2979 0 -1 4194304 23002 0 0 0 30097 18 0 0 25 0 1 0 1113312915 94470144 22697 1992294400 134512640 135995732 4294956016 18446744073709551615 134682179 0 0 4096 1 0 0 0 17 0 0 0
/proc/3303/statm: 23064 22697 172 362 0 22699 0
Current children cumulated CPU time (s) 1001.73
Current children cumulated vsize (KiB) 97620

[startup+1062.3 s]
/proc/loadavg: 1.00 1.00 1.02 2/74 3303
/proc/meminfo: memFree=817064/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5364 CPUtime=700.58
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 321 45096 0 0 0 0 69955 103 19 0 1 0 1113242817 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1341 245 202 169 0 52 0
[pid=3303] ppid=3298 vsize=116168 CPUtime=361.11
/proc/3303/stat : 3303 (hydrasat_cachew) R 3298 3298 2979 0 -1 4194304 28992 0 0 0 36089 22 0 0 25 0 1 0 1113312915 118956032 28687 1992294400 134512640 135995732 4294956016 18446744073709551615 134589575 0 0 4096 1 0 0 0 17 0 0 0
/proc/3303/statm: 29042 28687 172 362 0 28677 0
Current children cumulated CPU time (s) 1061.69
Current children cumulated vsize (KiB) 121532

[startup+1122.3 s]
/proc/loadavg: 1.00 1.00 1.01 2/74 3303
/proc/meminfo: memFree=817000/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5364 CPUtime=700.58
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 321 45096 0 0 0 0 69955 103 19 0 1 0 1113242817 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1341 245 202 169 0 52 0
[pid=3303] ppid=3298 vsize=116168 CPUtime=421.09
/proc/3303/stat : 3303 (hydrasat_cachew) R 3298 3298 2979 0 -1 4194304 28992 0 0 0 42086 23 0 0 25 0 1 0 1113312915 118956032 28687 1992294400 134512640 135995732 4294956016 18446744073709551615 134589481 0 0 4096 1 0 0 0 17 0 0 0
/proc/3303/statm: 29042 28687 172 362 0 28677 0
Current children cumulated CPU time (s) 1121.67
Current children cumulated vsize (KiB) 121532

[startup+1182.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/74 3303
/proc/meminfo: memFree=817000/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5364 CPUtime=700.58
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 321 45096 0 0 0 0 69955 103 19 0 1 0 1113242817 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1341 245 202 169 0 52 0
[pid=3303] ppid=3298 vsize=116168 CPUtime=481.06
/proc/3303/stat : 3303 (hydrasat_cachew) R 3298 3298 2979 0 -1 4194304 28992 0 0 0 48083 23 0 0 25 0 1 0 1113312915 118956032 28687 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 0 0 0
/proc/3303/statm: 29042 28687 172 362 0 28677 0
Current children cumulated CPU time (s) 1181.64
Current children cumulated vsize (KiB) 121532



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1200.7 s]
/proc/loadavg: 1.00 1.00 1.00 2/74 3303
/proc/meminfo: memFree=810280/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5364 CPUtime=700.58
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 321 45096 0 0 0 0 69955 103 19 0 1 0 1113242817 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1341 245 202 169 0 52 0
[pid=3303] ppid=3298 vsize=123016 CPUtime=499.45
/proc/3303/stat : 3303 (hydrasat_cachew) R 3298 3298 2979 0 -1 4194304 30687 0 0 0 49921 24 0 0 25 0 1 0 1113312915 125968384 30382 1992294400 134512640 135995732 4294956016 18446744073709551615 134807904 0 0 4096 1 0 0 0 17 0 0 0
/proc/3303/statm: 30754 30382 172 362 0 30389 0
Current children cumulated CPU time (s) 1200.03
Current children cumulated vsize (KiB) 128380

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1200.7 s]
/proc/loadavg: 1.00 1.00 1.00 2/74 3303
/proc/meminfo: memFree=810280/2055920 swapFree=4175896/4192956
[pid=3298] ppid=3296 vsize=5364 CPUtime=700.58
/proc/3298/stat : 3298 (hydrasat-multi.) S 3296 3298 2979 0 -1 4194304 321 45096 0 0 0 0 69955 103 19 0 1 0 1113242817 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 227607634756 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/3298/statm: 1341 245 202 169 0 52 0
[pid=3303] ppid=3298 vsize=123016 CPUtime=499.45
/proc/3303/stat : 3303 (hydrasat_cachew) R 3298 3298 2979 0 -1 4194304 30687 0 0 0 49921 24 0 0 25 0 1 0 1113312915 125968384 30382 1992294400 134512640 135995732 4294956016 18446744073709551615 134807904 0 0 4096 1 0 0 0 17 0 0 0
/proc/3303/statm: 30754 30382 172 362 0 30389 0
Current children cumulated CPU time (s) 1200.03
Current children cumulated vsize (KiB) 128380

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 3298 and gives
#  childrusage.ru_utime.tv_sec=699
#  childrusage.ru_utime.tv_usec=552651
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=33842
# CPU time returned by wait4() is 700.586
# while last known CPU time is 1200.03
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1200.71
CPU time (s): 1200.03
CPU user time (s): 1198.76
CPU system time (s): 1.27
CPU usage (%): 99.9431
Max. virtual memory (cumulated for all children) (KiB): 185192

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 699.553
system time used= 1.03384
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 45417
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5
involuntary context switches= 3008

runsolver used 1.10683 second user time and 3.33649 second system time

The end

Launcher Data 

Begin job on node66 at 2009-03-30 11:36:56
IDJOB=1529718
IDBENCH=70357
IDSOLVER=518
FILE ID=node66/1529718-1238405816
PBS_JOBID= 9061404
Free space on /tmp= 66540 MiB

SOLVER NAME= HydraSAT-Multi 2009-03-22-Multi
BENCH NAME= SAT09/RANDOM/LARGE/3SAT/UNKNOWN/18000/unif-k3-r4.2-v18000-c75600-S2088061175-062.cnf
COMMAND LINE= DIR/hydrasat-multi.sh DIR BENCHNAME MEMLIMIT RANDOMSEED
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-1529718-1238405816/watcher-1529718-1238405816 -o /tmp/evaluation-result-1529718-1238405816/solver-1529718-1238405816 -C 1200 -W 1800 -M 1800 --output-limit 1,15  HOME/hydrasat-multi.sh HOME HOME/instance-1529718-1238405816.cnf 1800 105379673

TIME LIMIT= 1200 seconds
MEMORY LIMIT= 1800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 0b73ddee1057244eb6a8d3ac4fd7781f
RANDOM SEED=105379673

node66.alineos.net Linux 2.6.9-22.EL.rootsmp #1 SMP Mon Oct 3 08:59:52 CEST 2005

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.286
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.286
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1109464 kB
Buffers:         93400 kB
Cached:         759676 kB
SwapCached:       8824 kB
Active:         257276 kB
Inactive:       606708 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1109464 kB
SwapTotal:     4192956 kB
SwapFree:      4175896 kB
Dirty:            5504 kB
Writeback:           0 kB
Mapped:          16576 kB
Slab:            67720 kB
Committed_AS:  3100308 kB
PageTables:       1976 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66364 MiB
End job on node66 at 2009-03-30 11:56:59