Trace number 3380861

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001.9 5012.87

General information on the benchmark

Nameapplication/jarvisalo/smtqfbv-aigs/
smtlib-qfbv-aigs-VS3-benchmark-S2-tseitin.cnf
MD5SUMb967fca11e85ef5b614d3452b1c4b2ad
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables257030
Number of clauses769313
Sum of the clauses size1795061
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2512874
Number of clauses of size 3256437
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...493979/instance-3380861-1305493979.cnf
2.09/2.13	c Solving...
40000.40/5011.53	
40000.40/5011.53	*** INTERRUPTED! ***
40000.40/5011.53	*** Shutting down threads...
40001.83/5012.85	s UNKNOWN
40001.83/5012.85	
40001.83/5012.85	c INTERRUPTED   : 1
40001.83/5012.85	c Models        : 0+    
40001.83/5012.85	c Time          : 5011.553s (Solving: 5009.42s 1st Model: 0.00s Unsat: 0.00s)
40001.83/5012.85	c CPU Time      : 40000.580s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3380861-1305493979/watcher-3380861-1305493979 -o /tmp/evaluation-result-3380861-1305493979/solver-3380861-1305493979 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380861-1305493979.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.10 4.12 4.35 1/417 350
/proc/meminfo: memFree=21166960/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=20792 CPUtime=0 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4194304 253 0 0 0 0 0 0 0 21 0 1 0 739628621 21291008 202 33554432000 4194304 4964800 140733235448032 18446744073709551615 260900431761 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/350/statm: 5198 202 167 189 0 59 0

[startup+0.0783919 s]
/proc/loadavg: 5.10 4.12 4.35 1/417 350
/proc/meminfo: memFree=21166960/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=77456 CPUtime=0.07 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 16855 0 0 0 6 1 0 0 21 0 1 0 739628621 79314944 14304 33554432000 4194304 4964800 140733235448032 18446744073709551615 260905054917 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/350/statm: 19364 14306 386 189 0 14238 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 77456

[startup+0.100388 s]
/proc/loadavg: 5.10 4.12 4.35 1/417 350
/proc/meminfo: memFree=21166960/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=83200 CPUtime=0.09 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 18336 0 0 0 8 1 0 0 21 0 1 0 739628621 85196800 15514 33554432000 4194304 4964800 140733235448032 18446744073709551615 260905091541 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/350/statm: 20800 15533 386 189 0 15674 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 83200

[startup+0.300352 s]
/proc/loadavg: 5.10 4.12 4.35 1/417 350
/proc/meminfo: memFree=21166960/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=122028 CPUtime=0.29 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 33511 0 0 0 26 3 0 0 22 0 1 0 739628621 124956672 24822 33554432000 4194304 4964800 140733235448032 18446744073709551615 260905091183 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/350/statm: 30507 24844 399 189 0 25381 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 122028

[startup+0.700278 s]
/proc/loadavg: 5.10 4.12 4.35 1/417 350
/proc/meminfo: memFree=21166960/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=153144 CPUtime=0.69 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 41049 0 0 0 65 4 0 0 24 0 1 0 739628621 156819456 32360 33554432000 4194304 4964800 140733235448032 18446744073709551615 260905054917 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/350/statm: 38286 32360 404 189 0 33160 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 153144

[startup+1.50013 s]
/proc/loadavg: 4.78 4.07 4.33 2/418 351
/proc/meminfo: memFree=21030552/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=172020 CPUtime=1.49 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 45763 0 0 0 144 5 0 0 25 0 1 0 739628621 176148480 37074 33554432000 4194304 4964800 140733235448032 18446744073709551615 260905054917 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/350/statm: 43005 37074 404 189 0 37879 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 172020

[startup+3.10317 s]
/proc/loadavg: 4.78 4.07 4.33 2/418 351
/proc/meminfo: memFree=21012448/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=683328 CPUtime=3.58 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 119535 0 0 0 327 31 0 0 18 0 8 0 739628621 699727872 92524 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668712 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/350/statm: 170832 92524 452 189 0 165181 0
Current children cumulated CPU time (s) 3.58
Current children cumulated vsize (KiB) 683328

[startup+6.30126 s]
/proc/loadavg: 4.78 4.07 4.33 11/425 359
/proc/meminfo: memFree=20789584/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=683328 CPUtime=29 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 122483 0 0 0 2866 34 0 0 25 0 8 0 739628621 699727872 95471 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668987 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/350/statm: 170832 95471 464 189 0 165181 0
[pid=350/tid=352] ppid=348 vsize=683328 CPUtime=3.28 cores=0-7
/proc/350/task/352/stat : 352 (claspmt) R 348 350 32630 0 -1 4202560 8695 0 0 0 325 3 0 0 20 0 8 0 739628834 699727872 95471 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=350/tid=353] ppid=348 vsize=683328 CPUtime=3.24 cores=0-7
/proc/350/task/353/stat : 353 (claspmt) R 348 350 32630 0 -1 4202560 10540 0 0 0 321 3 0 0 20 0 8 0 739628834 699727872 95471 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669044 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=350/tid=354] ppid=348 vsize=683328 CPUtime=3.29 cores=0-7
/proc/350/task/354/stat : 354 (claspmt) R 348 350 32630 0 -1 4202560 12461 0 0 0 324 5 0 0 19 0 8 0 739628834 699727872 95471 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=350/tid=355] ppid=348 vsize=683328 CPUtime=3.27 cores=0-7
/proc/350/task/355/stat : 355 (claspmt) R 348 350 32630 0 -1 4202560 9168 0 0 0 323 4 0 0 20 0 8 0 739628834 699727872 95471 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=350/tid=356] ppid=348 vsize=683328 CPUtime=4.04 cores=0-7
/proc/350/task/356/stat : 356 (claspmt) R 348 350 32630 0 -1 4202560 10454 0 0 0 401 3 0 0 25 0 8 0 739628834 699727872 95471 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668777 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=350/tid=357] ppid=348 vsize=683328 CPUtime=3.29 cores=0-7
/proc/350/task/357/stat : 357 (claspmt) R 348 350 32630 0 -1 4202560 12260 0 0 0 326 3 0 0 20 0 8 0 739628834 699727872 95471 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669280 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=350/tid=358] ppid=348 vsize=683328 CPUtime=3.32 cores=0-7
/proc/350/task/358/stat : 358 (claspmt) R 348 350 32630 0 -1 4202560 11052 0 0 0 327 5 0 0 20 0 8 0 739628834 699727872 95471 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 29
Current children cumulated vsize (KiB) 683328

[startup+12.7011 s]
/proc/loadavg: 5.42 4.23 4.38 9/425 359
/proc/meminfo: memFree=20779836/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=683328 CPUtime=80.1 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 125098 0 0 0 7972 38 0 0 25 0 8 0 739628621 699727872 98086 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670194 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/350/statm: 170832 98086 464 189 0 165181 0
[pid=350/tid=352] ppid=348 vsize=683328 CPUtime=9.67 cores=0-7
/proc/350/task/352/stat : 352 (claspmt) R 348 350 32630 0 -1 4202560 8800 0 0 0 964 3 0 0 25 0 8 0 739628834 699727872 98086 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=350/tid=353] ppid=348 vsize=683328 CPUtime=9.56 cores=0-7
/proc/350/task/353/stat : 353 (claspmt) R 348 350 32630 0 -1 4202560 11554 0 0 0 951 5 0 0 25 0 8 0 739628834 699727872 98086 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669044 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=350/tid=354] ppid=348 vsize=683328 CPUtime=9.69 cores=0-7
/proc/350/task/354/stat : 354 (claspmt) R 348 350 32630 0 -1 4202560 12835 0 0 0 964 5 0 0 25 0 8 0 739628834 699727872 98086 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=350/tid=355] ppid=348 vsize=683328 CPUtime=9.67 cores=0-7
/proc/350/task/355/stat : 355 (claspmt) R 348 350 32630 0 -1 4202560 9383 0 0 0 963 4 0 0 25 0 8 0 739628834 699727872 98086 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=350/tid=356] ppid=348 vsize=683328 CPUtime=10.44 cores=0-7
/proc/350/task/356/stat : 356 (claspmt) R 348 350 32630 0 -1 4202560 10460 0 0 0 1041 3 0 0 25 0 8 0 739628834 699727872 98086 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=350/tid=357] ppid=348 vsize=683328 CPUtime=9.69 cores=0-7
/proc/350/task/357/stat : 357 (claspmt) R 348 350 32630 0 -1 4202560 12474 0 0 0 966 3 0 0 25 0 8 0 739628834 699727872 98086 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=350/tid=358] ppid=348 vsize=683328 CPUtime=9.7 cores=0-7
/proc/350/task/358/stat : 358 (claspmt) R 348 350 32630 0 -1 4202560 11668 0 0 0 965 5 0 0 25 0 8 0 739628834 699727872 98086 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 80.1
Current children cumulated vsize (KiB) 683328

[startup+25.5007 s]
/proc/loadavg: 5.81 4.35 4.42 9/425 359
/proc/meminfo: memFree=20761988/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=683328 CPUtime=182.32 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 129777 0 0 0 18186 46 0 0 25 0 8 0 739628621 699727872 102765 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/350/statm: 170832 102765 464 189 0 165181 0
[pid=350/tid=352] ppid=348 vsize=683328 CPUtime=22.47 cores=0-7
/proc/350/task/352/stat : 352 (claspmt) R 348 350 32630 0 -1 4202560 8993 0 0 0 2244 3 0 0 25 0 8 0 739628834 699727872 102765 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668544 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=350/tid=353] ppid=348 vsize=683328 CPUtime=22.18 cores=0-7
/proc/350/task/353/stat : 353 (claspmt) R 348 350 32630 0 -1 4202560 13158 0 0 0 2211 7 0 0 25 0 8 0 739628834 699727872 102765 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670678 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=350/tid=354] ppid=348 vsize=683328 CPUtime=22.49 cores=0-7
/proc/350/task/354/stat : 354 (claspmt) R 348 350 32630 0 -1 4202560 13396 0 0 0 2243 6 0 0 25 0 8 0 739628834 699727872 102765 33554432000 4194304 4964800 140733235448032 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=350/tid=355] ppid=348 vsize=683328 CPUtime=22.47 cores=0-7
/proc/350/task/355/stat : 355 (claspmt) R 348 350 32630 0 -1 4202560 10061 0 0 0 2242 5 0 0 25 0 8 0 739628834 699727872 102765 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=350/tid=356] ppid=348 vsize=683328 CPUtime=23.24 cores=0-7
/proc/350/task/356/stat : 356 (claspmt) R 348 350 32630 0 -1 4202560 10513 0 0 0 2321 3 0 0 25 0 8 0 739628834 699727872 102765 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=350/tid=357] ppid=348 vsize=683328 CPUtime=22.49 cores=0-7
/proc/350/task/357/stat : 357 (claspmt) R 348 350 32630 0 -1 4202560 12700 0 0 0 2245 4 0 0 25 0 8 0 739628834 699727872 102765 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668737 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=350/tid=358] ppid=348 vsize=683328 CPUtime=22.51 cores=0-7
/proc/350/task/358/stat : 358 (claspmt) R 348 350 32630 0 -1 4202560 12892 0 0 0 2243 8 0 0 25 0 8 0 739628834 699727872 102765 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 182.32
Current children cumulated vsize (KiB) 683328

[startup+51.1011 s]
/proc/loadavg: 6.56 4.64 4.51 9/425 360
/proc/meminfo: memFree=20727540/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=689844 CPUtime=386.78 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 138409 0 0 0 38623 55 0 0 25 0 8 0 739628621 706400256 111367 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/350/statm: 172461 111367 466 189 0 166810 0
[pid=350/tid=352] ppid=348 vsize=689844 CPUtime=48.08 cores=0-7
/proc/350/task/352/stat : 352 (claspmt) R 348 350 32630 0 -1 4202560 9485 0 0 0 4804 4 0 0 25 0 8 0 739628834 706400256 111367 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=350/tid=353] ppid=348 vsize=689844 CPUtime=47.43 cores=0-7
/proc/350/task/353/stat : 353 (claspmt) R 348 350 32630 0 -1 4202560 15375 0 0 0 4734 9 0 0 25 0 8 0 739628834 706400256 111367 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=350/tid=354] ppid=348 vsize=689844 CPUtime=48.09 cores=0-7
/proc/350/task/354/stat : 354 (claspmt) R 348 350 32630 0 -1 4202560 13972 0 0 0 4803 6 0 0 25 0 8 0 739628834 706400256 111367 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=350/tid=355] ppid=348 vsize=689844 CPUtime=48.07 cores=0-7
/proc/350/task/355/stat : 355 (claspmt) R 348 350 32630 0 -1 4202560 11123 0 0 0 4801 6 0 0 25 0 8 0 739628834 706400256 111367 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=350/tid=356] ppid=348 vsize=689844 CPUtime=48.85 cores=0-7
/proc/350/task/356/stat : 356 (claspmt) R 348 350 32630 0 -1 4202560 11715 0 0 0 4880 5 0 0 25 0 8 0 739628834 706400256 111367 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=350/tid=357] ppid=348 vsize=689844 CPUtime=48.1 cores=0-7
/proc/350/task/357/stat : 357 (claspmt) R 348 350 32630 0 -1 4202560 13369 0 0 0 4805 5 0 0 25 0 8 0 739628834 706400256 111367 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=350/tid=358] ppid=348 vsize=689844 CPUtime=48.1 cores=0-7
/proc/350/task/358/stat : 358 (claspmt) R 348 350 32630 0 -1 4202560 15183 0 0 0 4800 10 0 0 25 0 8 0 739628834 706400256 111367 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 386.78
Current children cumulated vsize (KiB) 689844

[startup+102.317 s]
/proc/loadavg: 7.46 5.22 4.71 9/425 361
/proc/meminfo: memFree=20647540/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=698304 CPUtime=795.78 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 158316 0 0 0 79500 78 0 0 25 0 8 0 739628621 715063296 131274 33554432000 4194304 4964800 140733235448032 18446744073709551615 4684107 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/350/statm: 174576 131274 466 189 0 168925 0
[pid=350/tid=352] ppid=348 vsize=698304 CPUtime=99.29 cores=0-7
/proc/350/task/352/stat : 352 (claspmt) R 348 350 32630 0 -1 4202560 10782 0 0 0 9923 6 0 0 25 0 8 0 739628834 715063296 131274 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=350/tid=353] ppid=348 vsize=698304 CPUtime=97.93 cores=0-7
/proc/350/task/353/stat : 353 (claspmt) R 348 350 32630 0 -1 4202560 19421 0 0 0 9779 14 0 0 25 0 8 0 739628834 715063296 131274 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=350/tid=354] ppid=348 vsize=698304 CPUtime=99.31 cores=0-7
/proc/350/task/354/stat : 354 (claspmt) R 348 350 32630 0 -1 4202560 15559 0 0 0 9924 7 0 0 25 0 8 0 739628834 715063296 131274 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=350/tid=355] ppid=348 vsize=698304 CPUtime=99.28 cores=0-7
/proc/350/task/355/stat : 355 (claspmt) R 348 350 32630 0 -1 4202560 13621 0 0 0 9920 8 0 0 25 0 8 0 739628834 715063296 131274 33554432000 4194304 4964800 140733235448032 18446744073709551615 260905054858 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=350/tid=356] ppid=348 vsize=698304 CPUtime=100.06 cores=0-7
/proc/350/task/356/stat : 356 (claspmt) R 348 350 32630 0 -1 4202560 15261 0 0 0 9998 8 0 0 25 0 8 0 739628834 715063296 131274 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=350/tid=357] ppid=348 vsize=698304 CPUtime=99.31 cores=0-7
/proc/350/task/357/stat : 357 (claspmt) R 348 350 32630 0 -1 4202560 14233 0 0 0 9926 5 0 0 25 0 8 0 739628834 715063296 131274 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668715 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=350/tid=358] ppid=348 vsize=698304 CPUtime=99.3 cores=0-7
/proc/350/task/358/stat : 358 (claspmt) R 348 350 32630 0 -1 4202560 20455 0 0 0 9912 18 0 0 25 0 8 0 739628834 715063296 131274 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 795.78
Current children cumulated vsize (KiB) 698304

[startup+162.301 s]
/proc/loadavg: 7.91 5.75 4.93 9/425 363
/proc/meminfo: memFree=20583512/32950928 swapFree=65730872/67111528

################
# More data... #
################

[pid=350] ppid=348 vsize=2090112 CPUtime=37690.8 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 468963 0 0 0 3768600 481 0 0 25 0 8 0 739628621 2140274688 438810 33554432000 4194304 4964800 140733235448032 18446744073709551615 4321079 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/350/statm: 522528 438810 466 189 0 516877 0
[pid=350/tid=352] ppid=348 vsize=2090112 CPUtime=4719.33 cores=0-7
/proc/350/task/352/stat : 352 (claspmt) R 348 350 32630 0 -1 4202560 23669 0 0 0 471909 24 0 0 25 0 8 0 739628834 2140274688 438810 33554432000 4194304 4964800 140733235448032 18446744073709551615 4659391 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=350/tid=353] ppid=348 vsize=2090112 CPUtime=4673.2 cores=0-7
/proc/350/task/353/stat : 353 (claspmt) R 348 350 32630 0 -1 4202560 80768 0 0 0 467260 60 0 0 25 0 8 0 739628834 2140274688 438810 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=350/tid=354] ppid=348 vsize=2090112 CPUtime=4718.52 cores=0-7
/proc/350/task/354/stat : 354 (claspmt) R 348 350 32630 0 -1 4202560 30444 0 0 0 471831 21 0 0 25 0 8 0 739628834 2140274688 438810 33554432000 4194304 4964800 140733235448032 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=350/tid=355] ppid=348 vsize=2090112 CPUtime=4719.27 cores=0-7
/proc/350/task/355/stat : 355 (claspmt) R 348 350 32630 0 -1 4202560 35978 0 0 0 471890 37 0 0 25 0 8 0 739628834 2140274688 438810 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=350/tid=356] ppid=348 vsize=2090112 CPUtime=4720.18 cores=0-7
/proc/350/task/356/stat : 356 (claspmt) R 348 350 32630 0 -1 4202560 42230 0 0 0 471985 33 0 0 25 0 8 0 739628834 2140274688 438810 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=350/tid=357] ppid=348 vsize=2090112 CPUtime=4719.35 cores=0-7
/proc/350/task/357/stat : 357 (claspmt) R 348 350 32630 0 -1 4202560 28230 0 0 0 471912 23 0 0 25 0 8 0 739628834 2140274688 438810 33554432000 4194304 4964800 140733235448032 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=350/tid=358] ppid=348 vsize=2090112 CPUtime=4718.35 cores=0-7
/proc/350/task/358/stat : 358 (claspmt) R 348 350 32630 0 -1 4202560 168226 0 0 0 471588 247 0 0 25 0 8 0 739628834 2140274688 438810 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37690.8
Current children cumulated vsize (KiB) 2090112

[startup+4782.3 s]
/proc/loadavg: 8.07 8.06 8.00 9/425 508
/proc/meminfo: memFree=19409700/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=2090112 CPUtime=38169.9 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 469384 0 0 0 3816504 481 0 0 25 0 8 0 739628621 2140274688 439214 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/350/statm: 522528 439214 466 189 0 516877 0
[pid=350/tid=352] ppid=348 vsize=2090112 CPUtime=4779.32 cores=0-7
/proc/350/task/352/stat : 352 (claspmt) R 348 350 32630 0 -1 4202560 23669 0 0 0 477908 24 0 0 25 0 8 0 739628834 2140274688 439214 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669280 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=350/tid=353] ppid=348 vsize=2090112 CPUtime=4733.18 cores=0-7
/proc/350/task/353/stat : 353 (claspmt) R 348 350 32630 0 -1 4202560 81118 0 0 0 473258 60 0 0 25 0 8 0 739628834 2140274688 439214 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=350/tid=354] ppid=348 vsize=2090112 CPUtime=4778.5 cores=0-7
/proc/350/task/354/stat : 354 (claspmt) R 348 350 32630 0 -1 4202560 30444 0 0 0 477829 21 0 0 25 0 8 0 739628834 2140274688 439214 33554432000 4194304 4964800 140733235448032 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=350/tid=355] ppid=348 vsize=2090112 CPUtime=4779.26 cores=0-7
/proc/350/task/355/stat : 355 (claspmt) R 348 350 32630 0 -1 4202560 36013 0 0 0 477889 37 0 0 25 0 8 0 739628834 2140274688 439214 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=350/tid=356] ppid=348 vsize=2090112 CPUtime=4780.17 cores=0-7
/proc/350/task/356/stat : 356 (claspmt) R 348 350 32630 0 -1 4202560 42266 0 0 0 477984 33 0 0 25 0 8 0 739628834 2140274688 439214 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=350/tid=357] ppid=348 vsize=2090112 CPUtime=4779.33 cores=0-7
/proc/350/task/357/stat : 357 (claspmt) R 348 350 32630 0 -1 4202560 28230 0 0 0 477910 23 0 0 25 0 8 0 739628834 2140274688 439214 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=350/tid=358] ppid=348 vsize=2090112 CPUtime=4778.32 cores=0-7
/proc/350/task/358/stat : 358 (claspmt) R 348 350 32630 0 -1 4202560 168226 0 0 0 477585 247 0 0 25 0 8 0 739628834 2140274688 439214 33554432000 4194304 4964800 140733235448032 18446744073709551615 4342232 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38169.8
Current children cumulated vsize (KiB) 2090112

[startup+4842.3 s]
/proc/loadavg: 8.08 8.06 8.00 9/425 510
/proc/meminfo: memFree=19406540/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=2090112 CPUtime=38649 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 470207 0 0 0 3864420 482 0 0 25 0 8 0 739628621 2140274688 440037 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669011 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/350/statm: 522528 440037 466 189 0 516877 0
[pid=350/tid=352] ppid=348 vsize=2090112 CPUtime=4839.32 cores=0-7
/proc/350/task/352/stat : 352 (claspmt) R 348 350 32630 0 -1 4202560 23693 0 0 0 483908 24 0 0 25 0 8 0 739628834 2140274688 440037 33554432000 4194304 4964800 140733235448032 18446744073709551615 4320939 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=350/tid=353] ppid=348 vsize=2090112 CPUtime=4793.18 cores=0-7
/proc/350/task/353/stat : 353 (claspmt) R 348 350 32630 0 -1 4202560 81153 0 0 0 479258 60 0 0 25 0 8 0 739628834 2140274688 440037 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=350/tid=354] ppid=348 vsize=2090112 CPUtime=4838.51 cores=0-7
/proc/350/task/354/stat : 354 (claspmt) R 348 350 32630 0 -1 4202560 30444 0 0 0 483830 21 0 0 25 0 8 0 739628834 2140274688 440037 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=350/tid=355] ppid=348 vsize=2090112 CPUtime=4839.26 cores=0-7
/proc/350/task/355/stat : 355 (claspmt) R 348 350 32630 0 -1 4202560 36013 0 0 0 483889 37 0 0 25 0 8 0 739628834 2140274688 440037 33554432000 4194304 4964800 140733235448032 18446744073709551615 4362375 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=350/tid=356] ppid=348 vsize=2090112 CPUtime=4840.17 cores=0-7
/proc/350/task/356/stat : 356 (claspmt) R 348 350 32630 0 -1 4202560 42272 0 0 0 483984 33 0 0 25 0 8 0 739628834 2140274688 440037 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=350/tid=357] ppid=348 vsize=2090112 CPUtime=4839.33 cores=0-7
/proc/350/task/357/stat : 357 (claspmt) R 348 350 32630 0 -1 4202560 28230 0 0 0 483909 24 0 0 25 0 8 0 739628834 2140274688 440037 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=350/tid=358] ppid=348 vsize=2090112 CPUtime=4838.31 cores=0-7
/proc/350/task/358/stat : 358 (claspmt) R 348 350 32630 0 -1 4202560 168984 0 0 0 483583 248 0 0 25 0 8 0 739628834 2140274688 440037 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38649
Current children cumulated vsize (KiB) 2090112

[startup+4902.3 s]
/proc/loadavg: 8.07 8.06 8.00 9/425 511
/proc/meminfo: memFree=19403872/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=2090112 CPUtime=39128.2 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 470847 0 0 0 3912338 483 0 0 25 0 8 0 739628621 2140274688 440677 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670499 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/350/statm: 522528 440677 466 189 0 516877 0
[pid=350/tid=352] ppid=348 vsize=2090112 CPUtime=4899.32 cores=0-7
/proc/350/task/352/stat : 352 (claspmt) R 348 350 32630 0 -1 4202560 23703 0 0 0 489908 24 0 0 25 0 8 0 739628834 2140274688 440677 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=350/tid=353] ppid=348 vsize=2090112 CPUtime=4853.18 cores=0-7
/proc/350/task/353/stat : 353 (claspmt) R 348 350 32630 0 -1 4202560 81763 0 0 0 485258 60 0 0 25 0 8 0 739628834 2140274688 440677 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=350/tid=354] ppid=348 vsize=2090112 CPUtime=4898.51 cores=0-7
/proc/350/task/354/stat : 354 (claspmt) R 348 350 32630 0 -1 4202560 30444 0 0 0 489830 21 0 0 25 0 8 0 739628834 2140274688 440677 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=350/tid=355] ppid=348 vsize=2090112 CPUtime=4899.26 cores=0-7
/proc/350/task/355/stat : 355 (claspmt) R 348 350 32630 0 -1 4202560 36013 0 0 0 489889 37 0 0 25 0 8 0 739628834 2140274688 440677 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=350/tid=356] ppid=348 vsize=2090112 CPUtime=4900.17 cores=0-7
/proc/350/task/356/stat : 356 (claspmt) R 348 350 32630 0 -1 4202560 42272 0 0 0 489984 33 0 0 25 0 8 0 739628834 2140274688 440677 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=350/tid=357] ppid=348 vsize=2090112 CPUtime=4899.34 cores=0-7
/proc/350/task/357/stat : 357 (claspmt) R 348 350 32630 0 -1 4202560 28247 0 0 0 489910 24 0 0 25 0 8 0 739628834 2140274688 440677 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669280 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=350/tid=358] ppid=348 vsize=2090112 CPUtime=4898.3 cores=0-7
/proc/350/task/358/stat : 358 (claspmt) R 348 350 32630 0 -1 4202560 168984 0 0 0 489582 248 0 0 25 0 8 0 739628834 2140274688 440677 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39128.2
Current children cumulated vsize (KiB) 2090112

[startup+4962.3 s]
/proc/loadavg: 8.11 8.08 8.01 9/425 513
/proc/meminfo: memFree=19402932/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=2090112 CPUtime=39607.4 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 471095 0 0 0 3960251 484 0 0 25 0 8 0 739628621 2140274688 440925 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669011 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/350/statm: 522528 440925 466 189 0 516877 0
[pid=350/tid=352] ppid=348 vsize=2090112 CPUtime=4959.33 cores=0-7
/proc/350/task/352/stat : 352 (claspmt) R 348 350 32630 0 -1 4202560 23764 0 0 0 495909 24 0 0 25 0 8 0 739628834 2140274688 440925 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=350/tid=353] ppid=348 vsize=2090112 CPUtime=4913.19 cores=0-7
/proc/350/task/353/stat : 353 (claspmt) R 348 350 32630 0 -1 4202560 81905 0 0 0 491258 61 0 0 25 0 8 0 739628834 2140274688 440925 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670349 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=350/tid=354] ppid=348 vsize=2090112 CPUtime=4958.51 cores=0-7
/proc/350/task/354/stat : 354 (claspmt) R 348 350 32630 0 -1 4202560 30456 0 0 0 495830 21 0 0 25 0 8 0 739628834 2140274688 440925 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=350/tid=355] ppid=348 vsize=2090112 CPUtime=4959.26 cores=0-7
/proc/350/task/355/stat : 355 (claspmt) R 348 350 32630 0 -1 4202560 36013 0 0 0 495889 37 0 0 25 0 8 0 739628834 2140274688 440925 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=350/tid=356] ppid=348 vsize=2090112 CPUtime=4960.17 cores=0-7
/proc/350/task/356/stat : 356 (claspmt) R 348 350 32630 0 -1 4202560 42287 0 0 0 495984 33 0 0 25 0 8 0 739628834 2140274688 440925 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=350/tid=357] ppid=348 vsize=2090112 CPUtime=4959.34 cores=0-7
/proc/350/task/357/stat : 357 (claspmt) R 348 350 32630 0 -1 4202560 28264 0 0 0 495910 24 0 0 25 0 8 0 739628834 2140274688 440925 33554432000 4194304 4964800 140733235448032 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=350/tid=358] ppid=348 vsize=2090112 CPUtime=4958.28 cores=0-7
/proc/350/task/358/stat : 358 (claspmt) R 348 350 32630 0 -1 4202560 168985 0 0 0 495580 248 0 0 25 0 8 0 739628834 2140274688 440925 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668715 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39607.4
Current children cumulated vsize (KiB) 2090112



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.5 s]
/proc/loadavg: 8.09 8.08 8.01 10/425 514
/proc/meminfo: memFree=19387244/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=2090112 CPUtime=40000.4 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 475161 0 0 0 3999549 491 0 0 25 0 8 0 739628621 2140274688 444991 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670346 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/350/statm: 522528 444991 466 189 0 516877 0
[pid=350/tid=352] ppid=348 vsize=2090112 CPUtime=5008.57 cores=0-7
/proc/350/task/352/stat : 352 (claspmt) R 348 350 32630 0 -1 4202560 23819 0 0 0 500832 25 0 0 25 0 8 0 739628834 2140274688 444991 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=350/tid=353] ppid=348 vsize=2090112 CPUtime=4962.42 cores=0-7
/proc/350/task/353/stat : 353 (claspmt) R 348 350 32630 0 -1 4202560 82477 0 0 0 496181 61 0 0 25 0 8 0 739628834 2140274688 444991 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=350/tid=354] ppid=348 vsize=2090112 CPUtime=5007.74 cores=0-7
/proc/350/task/354/stat : 354 (claspmt) R 348 350 32630 0 -1 4202560 30461 0 0 0 500753 21 0 0 25 0 8 0 739628834 2140274688 444991 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=350/tid=355] ppid=348 vsize=2090112 CPUtime=5008.49 cores=0-7
/proc/350/task/355/stat : 355 (claspmt) R 348 350 32630 0 -1 4202560 36013 0 0 0 500812 37 0 0 25 0 8 0 739628834 2140274688 444991 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=350/tid=356] ppid=348 vsize=2090112 CPUtime=5009.41 cores=0-7
/proc/350/task/356/stat : 356 (claspmt) R 348 350 32630 0 -1 4202560 42327 0 0 0 500908 33 0 0 25 0 8 0 739628834 2140274688 444991 33554432000 4194304 4964800 140733235448032 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=350/tid=357] ppid=348 vsize=2090112 CPUtime=5008.57 cores=0-7
/proc/350/task/357/stat : 357 (claspmt) R 348 350 32630 0 -1 4202560 28265 0 0 0 500833 24 0 0 25 0 8 0 739628834 2140274688 444991 33554432000 4194304 4964800 140733235448032 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=350/tid=358] ppid=348 vsize=2090112 CPUtime=5007.5 cores=0-7
/proc/350/task/358/stat : 358 (claspmt) R 348 350 32630 0 -1 4202560 171918 0 0 0 500496 254 0 0 25 0 8 0 739628834 2140274688 444991 33554432000 4194304 4964800 140733235448032 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 2090112

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011.8 s]
/proc/loadavg: 8.09 8.08 8.01 10/425 514
/proc/meminfo: memFree=19387244/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=2057728 CPUtime=40000.8 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 475173 0 0 0 3999592 491 0 0 25 0 1 0 739628621 2107113472 436615 33554432000 4194304 4964800 140733235448032 18446744073709551615 260905050011 0 0 4096 18946 0 0 0 17 7 0 0 0
/proc/350/statm: 514432 436615 476 189 0 508781 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 2057728

[startup+5012.6 s]
/proc/loadavg: 8.09 8.08 8.01 2/419 515
/proc/meminfo: memFree=19482152/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=1997468 CPUtime=40001.6 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 475173 0 0 0 3999672 491 0 0 25 0 1 0 739628621 2045407232 421037 33554432000 4194304 4964800 140733235448032 18446744073709551615 260905050673 0 0 4096 18946 0 0 0 17 7 0 0 0
/proc/350/statm: 499367 421037 476 189 0 493716 0
Current children cumulated CPU time (s) 40001.6
Current children cumulated vsize (KiB) 1997468

[startup+5012.8 s]
/proc/loadavg: 8.09 8.08 8.01 2/419 515
/proc/meminfo: memFree=19482152/32950928 swapFree=65730872/67111528
[pid=350] ppid=348 vsize=1961060 CPUtime=40001.8 cores=0-7
/proc/350/stat : 350 (claspmt) R 348 350 32630 0 -1 4202496 475173 0 0 0 3999691 492 0 0 25 0 1 0 739628621 2008125440 392360 33554432000 4194304 4964800 140733235448032 18446744073709551615 260905051922 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/350/statm: 490265 392360 476 189 0 484614 0
Current children cumulated CPU time (s) 40001.8
Current children cumulated vsize (KiB) 1961060

Child status: 0
Real time (s): 5012.87
CPU time (s): 40001.9
CPU user time (s): 39996.9
CPU system time (s): 4.98824
CPU usage (%): 797.984
Max. virtual memory (cumulated for all children) (KiB): 2090112

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39996.9
system time used= 4.98824
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 475174
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 37435
involuntary context switches= 187443

runsolver used 17.5633 second user time and 51.5542 second system time

The end

Launcher Data 

Begin job on node116 at 2011-05-15 23:12:59
IDJOB=3380861
IDBENCH=82459
IDSOLVER=1865
FILE ID=node116/3380861-1305493979
RUNJOBID= node116-1305493714-32644
PBS_JOBID= 13325428
Free space on /tmp= 73124 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/jarvisalo/smtqfbv-aigs/smtlib-qfbv-aigs-VS3-benchmark-S2-tseitin.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380861-1305493979/watcher-3380861-1305493979 -o /tmp/evaluation-result-3380861-1305493979/solver-3380861-1305493979 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380861-1305493979.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= b967fca11e85ef5b614d3452b1c4b2ad
RANDOM SEED=928567978

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      21167232 kB
Buffers:        150468 kB
Cached:        1356148 kB
SwapCached:      12348 kB
Active:       10574608 kB
Inactive:       969568 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      21167232 kB
SwapTotal:    67111528 kB
SwapFree:     65730872 kB
Dirty:           14648 kB
Writeback:           0 kB
AnonPages:    10036488 kB
Mapped:          19780 kB
Slab:           114388 kB
PageTables:      52704 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 113393244 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73124 MiB
End job on node116 at 2011-05-16 00:36:33