Trace number 1721759

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
HydraSAT-Multi 2009-03-22-Multi? (TO) 1200.07 1200.71

General information on the benchmark

NameAPPLICATIONS/satComp09_BioInstances/
rpoc_xits_13_UNKNOWN.cnf
MD5SUM57b85ec7870f670f13b00667a6315c05
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2058
Number of clauses144294
Sum of the clauses size429916
Maximum clause length14
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 23219
Number of clauses of size 3141052
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 523

Solver Data 

0.00/0.00	c Trying configuration 1
699.92/700.35	HOME/hydrasat-multi.sh: line 48: 18626 CPU time limit exceeded (core dumped) $SOLVER1 $BENCHNAME $OPTIONS1
699.92/700.35	c Trying configuration 2

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-1721759-1240669203/watcher-1721759-1240669203 -o /tmp/evaluation-result-1721759-1240669203/solver-1721759-1240669203 -C 1200 -W 1800 -M 1800 --output-limit 1,15 HOME/hydrasat-multi.sh HOME HOME/instance-1721759-1240669203.cnf 1800 1676268150 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1200 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1230 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 1843200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 1894400 KiB
Solver output will be limited to a maximum of 15728640 bytes. The first 1048576 bytes and the last 14680064 bytes will be preserved
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.92 0.98 1.07 3/64 18625
/proc/meminfo: memFree=1204704/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=2016 CPUtime=0
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 172 0 0 0 0 0 0 0 19 0 1 0 94954387 2064384 154 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 504 154 117 317 0 184 0

[startup+0.0737661 s]
/proc/loadavg: 0.92 0.98 1.07 3/64 18625
/proc/meminfo: memFree=1204704/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=5316 CPUtime=0.06
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 1051 0 0 0 6 0 0 0 19 0 1 0 94954387 5443584 1033 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 1329 1033 117 317 0 1009 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 10676

[startup+0.102771 s]
/proc/loadavg: 0.92 0.98 1.07 3/64 18625
/proc/meminfo: memFree=1204704/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=6768 CPUtime=0.09
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 1422 0 0 0 9 0 0 0 19 0 1 0 94954387 6930432 1404 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 1692 1405 117 317 0 1372 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 12128

[startup+0.301814 s]
/proc/loadavg: 0.92 0.98 1.07 3/64 18625
/proc/meminfo: memFree=1204704/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=13460 CPUtime=0.29
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 3294 0 0 0 28 1 0 0 21 0 1 0 94954387 13783040 3049 1992294400 134512640 135813592 4294956160 18446744073709551615 134676855 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 3365 3049 124 317 0 3045 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 18820

[startup+0.70188 s]
/proc/loadavg: 0.92 0.98 1.07 3/64 18625
/proc/meminfo: memFree=1204704/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=13460 CPUtime=0.69
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 3294 0 0 0 68 1 0 0 25 0 1 0 94954387 13783040 3049 1992294400 134512640 135813592 4294956160 18446744073709551615 134676855 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 3365 3049 124 317 0 3045 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 18820

[startup+1.50203 s]
/proc/loadavg: 0.92 0.98 1.07 2/66 18627
/proc/meminfo: memFree=1192528/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=14024 CPUtime=1.49
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 3439 0 0 0 148 1 0 0 25 0 1 0 94954387 14360576 3053 1992294400 134512640 135813592 4294956160 18446744073709551615 134521120 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 3506 3053 126 317 0 3186 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 19384

[startup+3.10132 s]
/proc/loadavg: 0.92 0.98 1.07 2/66 18627
/proc/meminfo: memFree=1189656/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=16928 CPUtime=3.08
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 4175 0 0 0 307 1 0 0 25 0 1 0 94954387 17334272 3789 1992294400 134512640 135813592 4294956160 18446744073709551615 134521412 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 4232 3789 126 317 0 3912 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 22288

[startup+6.3019 s]
/proc/loadavg: 1.01 1.00 1.08 2/66 18627
/proc/meminfo: memFree=1184536/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=22636 CPUtime=6.29
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 5626 0 0 0 627 2 0 0 25 0 1 0 94954387 23179264 5102 1992294400 134512640 135813592 4294956160 18446744073709551615 135205332 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 5659 5102 149 317 0 5339 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 27996

[startup+12.7011 s]
/proc/loadavg: 1.01 1.00 1.08 2/66 18627
/proc/meminfo: memFree=1183328/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=23584 CPUtime=12.68
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 5876 0 0 0 1266 2 0 0 25 0 1 0 94954387 24150016 5352 1992294400 134512640 135813592 4294956160 18446744073709551615 134665562 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 5896 5352 149 317 0 5576 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 28944

[startup+25.5024 s]
/proc/loadavg: 1.00 1.00 1.08 2/66 18627
/proc/meminfo: memFree=1182240/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=25112 CPUtime=25.47
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 6183 0 0 0 2545 2 0 0 25 0 1 0 94954387 25714688 5659 1992294400 134512640 135813592 4294956160 18446744073709551615 134665279 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 6278 5659 149 317 0 5958 0
Current children cumulated CPU time (s) 25.47
Current children cumulated vsize (KiB) 30472

[startup+51.1021 s]
/proc/loadavg: 1.00 1.00 1.07 2/66 18627
/proc/meminfo: memFree=1181920/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=25524 CPUtime=51.06
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 6270 0 0 0 5104 2 0 0 25 0 1 0 94954387 26136576 5746 1992294400 134512640 135813592 4294956160 18446744073709551615 134665049 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 6381 5746 149 317 0 6061 0
Current children cumulated CPU time (s) 51.06
Current children cumulated vsize (KiB) 30884

[startup+102.305 s]
/proc/loadavg: 1.00 1.00 1.07 2/66 18627
/proc/meminfo: memFree=1181408/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=26088 CPUtime=102.24
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 6381 0 0 0 10222 2 0 0 25 0 1 0 94954387 26714112 5857 1992294400 134512640 135813592 4294956160 18446744073709551615 134665049 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 6522 5857 149 317 0 6202 0
Current children cumulated CPU time (s) 102.24
Current children cumulated vsize (KiB) 31448

[startup+162.302 s]
/proc/loadavg: 1.00 1.00 1.06 2/66 18627
/proc/meminfo: memFree=1181280/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=26228 CPUtime=162.2
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 6388 0 0 0 16218 2 0 0 25 0 1 0 94954387 26857472 5864 1992294400 134512640 135813592 4294956160 18446744073709551615 134665006 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 6557 5864 149 317 0 6237 0
Current children cumulated CPU time (s) 162.2
Current children cumulated vsize (KiB) 31588

[startup+222.301 s]
/proc/loadavg: 1.00 1.00 1.06 2/66 18627
/proc/meminfo: memFree=1181152/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=26360 CPUtime=222.16
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 6420 0 0 0 22214 2 0 0 25 0 1 0 94954387 26992640 5896 1992294400 134512640 135813592 4294956160 18446744073709551615 134665557 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 6590 5896 149 317 0 6270 0
Current children cumulated CPU time (s) 222.16
Current children cumulated vsize (KiB) 31720

[startup+282.301 s]
/proc/loadavg: 1.00 1.00 1.05 2/66 18627
/proc/meminfo: memFree=1181024/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=26492 CPUtime=282.14
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 6455 0 0 0 28211 3 0 0 25 0 1 0 94954387 27127808 5931 1992294400 134512640 135813592 4294956160 18446744073709551615 134664877 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 6623 5931 149 317 0 6303 0
Current children cumulated CPU time (s) 282.14
Current children cumulated vsize (KiB) 31852

[startup+342.301 s]
/proc/loadavg: 1.00 1.00 1.05 2/66 18627
/proc/meminfo: memFree=1180832/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=26640 CPUtime=342.11
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 6507 0 0 0 34208 3 0 0 25 0 1 0 94954387 27279360 5983 1992294400 134512640 135813592 4294956160 18446744073709551615 134665537 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 6660 5983 149 317 0 6340 0

################
# More data... #
################

/proc/18626/statm: 6981 6245 149 317 0 6661 0
Current children cumulated CPU time (s) 581.98
Current children cumulated vsize (KiB) 33284

[startup+642.301 s]
/proc/loadavg: 1.00 1.00 1.02 2/66 18627
/proc/meminfo: memFree=1179744/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5360 CPUtime=0
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 94954386 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1340 239 197 169 0 51 0
[pid=18626] ppid=18625 vsize=27924 CPUtime=641.96
/proc/18626/stat : 18626 (hydrasat_cache4) R 18625 18625 17772 0 -1 4194304 6769 0 0 0 64193 3 0 0 25 0 1 0 94954387 28594176 6245 1992294400 134512640 135813592 4294956160 18446744073709551615 134662639 0 0 4096 1 0 0 0 17 0 0 0
/proc/18626/statm: 6981 6245 149 317 0 6661 0
Current children cumulated CPU time (s) 641.96
Current children cumulated vsize (KiB) 33284

[startup+702.301 s]
/proc/loadavg: 1.00 1.00 1.01 2/66 18628
/proc/meminfo: memFree=1169312/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5364 CPUtime=699.98
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 321 7336 0 0 0 0 69985 13 19 0 1 0 94954386 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1341 245 202 169 0 52 0
[pid=18628] ppid=18625 vsize=14856 CPUtime=1.92
/proc/18628/stat : 18628 (hydrasat_cachew) R 18625 18625 17772 0 -1 4194304 3627 0 0 0 191 1 0 0 25 0 1 0 95024422 15212544 3242 1992294400 134512640 135995732 4294956016 18446744073709551615 134520024 0 0 4096 1 0 0 0 17 0 0 0
/proc/18628/statm: 3714 3242 150 362 0 3349 0
Current children cumulated CPU time (s) 701.9
Current children cumulated vsize (KiB) 20220

[startup+762.305 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18628
/proc/meminfo: memFree=1164576/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5364 CPUtime=699.98
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 321 7336 0 0 0 0 69985 13 19 0 1 0 94954386 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1341 245 202 169 0 52 0
[pid=18628] ppid=18625 vsize=17368 CPUtime=61.9
/proc/18628/stat : 18628 (hydrasat_cachew) R 18625 18625 17772 0 -1 4194304 4408 0 0 0 6188 2 0 0 25 0 1 0 95024422 17784832 3885 1992294400 134512640 135995732 4294956016 18446744073709551615 134806544 0 0 4096 1 0 0 0 17 0 0 0
/proc/18628/statm: 4342 3885 171 362 0 3977 0
Current children cumulated CPU time (s) 761.88
Current children cumulated vsize (KiB) 22732

[startup+822.301 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18628
/proc/meminfo: memFree=1162912/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5364 CPUtime=699.98
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 321 7336 0 0 0 0 69985 13 19 0 1 0 94954386 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1341 245 202 169 0 52 0
[pid=18628] ppid=18625 vsize=19088 CPUtime=121.88
/proc/18628/stat : 18628 (hydrasat_cachew) R 18625 18625 17772 0 -1 4194304 4824 0 0 0 12185 3 0 0 25 0 1 0 95024422 19546112 4301 1992294400 134512640 135995732 4294956016 18446744073709551615 134588962 0 0 4096 1 0 0 0 17 0 0 0
/proc/18628/statm: 4772 4301 171 362 0 4407 0
Current children cumulated CPU time (s) 821.86
Current children cumulated vsize (KiB) 24452

[startup+882.301 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18628
/proc/meminfo: memFree=1158752/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5364 CPUtime=699.98
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 321 7336 0 0 0 0 69985 13 19 0 1 0 94954386 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1341 245 202 169 0 52 0
[pid=18628] ppid=18625 vsize=23544 CPUtime=181.84
/proc/18628/stat : 18628 (hydrasat_cachew) R 18625 18625 17772 0 -1 4194304 5898 0 0 0 18180 4 0 0 25 0 1 0 95024422 24109056 5375 1992294400 134512640 135995732 4294956016 18446744073709551615 134807828 0 0 4096 1 0 0 0 17 0 0 0
/proc/18628/statm: 5886 5375 171 362 0 5521 0
Current children cumulated CPU time (s) 881.82
Current children cumulated vsize (KiB) 28908

[startup+942.301 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18628
/proc/meminfo: memFree=1157344/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5364 CPUtime=699.98
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 321 7336 0 0 0 0 69985 13 19 0 1 0 94954386 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1341 245 202 169 0 52 0
[pid=18628] ppid=18625 vsize=25144 CPUtime=241.81
/proc/18628/stat : 18628 (hydrasat_cachew) R 18625 18625 17772 0 -1 4194304 6257 0 0 0 24177 4 0 0 25 0 1 0 95024422 25747456 5734 1992294400 134512640 135995732 4294956016 18446744073709551615 134806628 0 0 4096 1 0 0 0 17 0 0 0
/proc/18628/statm: 6286 5734 171 362 0 5921 0
Current children cumulated CPU time (s) 941.79
Current children cumulated vsize (KiB) 30508

[startup+1002.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18628
/proc/meminfo: memFree=1153824/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5364 CPUtime=699.98
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 321 7336 0 0 0 0 69985 13 19 0 1 0 94954386 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1341 245 202 169 0 52 0
[pid=18628] ppid=18625 vsize=28888 CPUtime=301.78
/proc/18628/stat : 18628 (hydrasat_cachew) R 18625 18625 17772 0 -1 4194304 7140 0 0 0 30173 5 0 0 25 0 1 0 95024422 29581312 6617 1992294400 134512640 135995732 4294956016 18446744073709551615 134807874 0 0 4096 1 0 0 0 17 0 0 0
/proc/18628/statm: 7222 6617 171 362 0 6857 0
Current children cumulated CPU time (s) 1001.76
Current children cumulated vsize (KiB) 34252

[startup+1062.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18630
/proc/meminfo: memFree=1151840/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5364 CPUtime=699.98
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 321 7336 0 0 0 0 69985 13 19 0 1 0 94954386 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1341 245 202 169 0 52 0
[pid=18628] ppid=18625 vsize=30880 CPUtime=361.75
/proc/18628/stat : 18628 (hydrasat_cachew) R 18625 18625 17772 0 -1 4194304 7609 0 0 0 36169 6 0 0 25 0 1 0 95024422 31621120 7086 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 0 0 0
/proc/18628/statm: 7720 7086 171 362 0 7355 0
Current children cumulated CPU time (s) 1061.73
Current children cumulated vsize (KiB) 36244

[startup+1122.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18630
/proc/meminfo: memFree=1148704/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5364 CPUtime=699.98
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 321 7336 0 0 0 0 69985 13 19 0 1 0 94954386 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1341 245 202 169 0 52 0
[pid=18628] ppid=18625 vsize=34404 CPUtime=421.73
/proc/18628/stat : 18628 (hydrasat_cachew) R 18625 18625 17772 0 -1 4194304 8384 0 0 0 42166 7 0 0 25 0 1 0 95024422 35229696 7861 1992294400 134512640 135995732 4294956016 18446744073709551615 134807701 0 0 4096 1 0 0 0 17 0 0 0
/proc/18628/statm: 8601 7861 171 362 0 8236 0
Current children cumulated CPU time (s) 1121.71
Current children cumulated vsize (KiB) 39768

[startup+1182.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18630
/proc/meminfo: memFree=1141280/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5364 CPUtime=699.98
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 321 7336 0 0 0 0 69985 13 19 0 1 0 94954386 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1341 245 202 169 0 52 0
[pid=18628] ppid=18625 vsize=42068 CPUtime=481.7
/proc/18628/stat : 18628 (hydrasat_cachew) R 18625 18625 17772 0 -1 4194304 10238 0 0 0 48161 9 0 0 25 0 1 0 95024422 43077632 9715 1992294400 134512640 135995732 4294956016 18446744073709551615 134807860 0 0 4096 1 0 0 0 17 0 0 0
/proc/18628/statm: 10517 9715 171 362 0 10152 0
Current children cumulated CPU time (s) 1181.68
Current children cumulated vsize (KiB) 47432



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1200.7 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18630
/proc/meminfo: memFree=1140896/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5364 CPUtime=699.98
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 321 7336 0 0 0 0 69985 13 19 0 1 0 94954386 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1341 245 202 169 0 52 0
[pid=18628] ppid=18625 vsize=42680 CPUtime=500.09
/proc/18628/stat : 18628 (hydrasat_cachew) R 18625 18625 17772 0 -1 4194304 10329 0 0 0 50000 9 0 0 25 0 1 0 95024422 43704320 9806 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 0 0 0
/proc/18628/statm: 10670 9806 171 362 0 10305 0
Current children cumulated CPU time (s) 1200.07
Current children cumulated vsize (KiB) 48044

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1200.7 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18630
/proc/meminfo: memFree=1140896/2055920 swapFree=4124000/4192956
[pid=18625] ppid=18623 vsize=5364 CPUtime=699.98
/proc/18625/stat : 18625 (hydrasat-multi.) S 18623 18625 17772 0 -1 4194304 321 7336 0 0 0 0 69985 13 19 0 1 0 94954386 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 206381310788 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18625/statm: 1341 245 202 169 0 52 0
[pid=18628] ppid=18625 vsize=42680 CPUtime=500.09
/proc/18628/stat : 18628 (hydrasat_cachew) R 18625 18625 17772 0 -1 4194304 10329 0 0 0 50000 9 0 0 25 0 1 0 95024422 43704320 9806 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 0 0 0
/proc/18628/statm: 10670 9806 171 362 0 10305 0
Current children cumulated CPU time (s) 1200.07
Current children cumulated vsize (KiB) 48044

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 18625 and gives
#  childrusage.ru_utime.tv_sec=699
#  childrusage.ru_utime.tv_usec=856605
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=133979
# CPU time returned by wait4() is 699.991
# while last known CPU time is 1200.07
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1200.71
CPU time (s): 1200.07
CPU user time (s): 1199.85
CPU system time (s): 0.22
CPU usage (%): 99.9466
Max. virtual memory (cumulated for all children) (KiB): 48044

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 699.857
system time used= 0.133979
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 7657
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5
involuntary context switches= 2039

runsolver used 1.11583 second user time and 3.2595 second system time

The end

Launcher Data 

Begin job on node2 at 2009-04-25 16:20:03
IDJOB=1721759
IDBENCH=71076
IDSOLVER=518
FILE ID=node2/1721759-1240669203
PBS_JOBID= 9186555
Free space on /tmp= 66312 MiB

SOLVER NAME= HydraSAT-Multi 2009-03-22-Multi
BENCH NAME= SAT09/APPLICATIONS/satComp09_BioInstances/rpoc_xits_13_UNKNOWN.cnf
COMMAND LINE= DIR/hydrasat-multi.sh DIR BENCHNAME MEMLIMIT RANDOMSEED
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-1721759-1240669203/watcher-1721759-1240669203 -o /tmp/evaluation-result-1721759-1240669203/solver-1721759-1240669203 -C 1200 -W 1800 -M 1800 --output-limit 1,15  HOME/hydrasat-multi.sh HOME HOME/instance-1721759-1240669203.cnf 1800 1676268150

TIME LIMIT= 1200 seconds
MEMORY LIMIT= 1800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 57b85ec7870f670f13b00667a6315c05
RANDOM SEED=1676268150

node2.alineos.net Linux 2.6.9-22.EL.rootsmp #1 SMP Mon Oct 3 08:59:52 CEST 2005

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.265
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.265
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1205184 kB
Buffers:         75376 kB
Cached:         633424 kB
SwapCached:      63068 kB
Active:         311520 kB
Inactive:       462784 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1205184 kB
SwapTotal:     4192956 kB
SwapFree:      4124000 kB
Dirty:            6372 kB
Writeback:           0 kB
Mapped:          13992 kB
Slab:            62396 kB
Committed_AS:   347880 kB
PageTables:       1468 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66288 MiB
End job on node2 at 2009-04-25 16:40:05