Trace number 3385840

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 4800.36 1205.1

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m37-q505.75-n35-p15-s48276711.cnf
MD5SUM1bac7b96ea6eb843b1de7405a721cd87
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.861868
Satisfiable
(Un)Satisfiability was proved
Number of variables1244
Number of clauses150604
Sum of the clauses size302382
Maximum clause length37
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2150569
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 535

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385840-1305583583/watcher-3385840-1305583583 -o /tmp/evaluation-result-3385840-1305583583/solver-3385840-1305583583 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3385840-1305583583.cnf -threads=4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.07 8.03 6.78 5/493 31770
/proc/meminfo: memFree=19336832/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=23284 CPUtime=0.01 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 1184 0 0 0 1 0 0 0 25 0 1 0 650774598 23842816 624 33554432000 4194304 4625164 140733678394240 18446744073709551615 212039655168 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/31770/statm: 5821 624 296 106 0 1324 0

[startup+0.164759 s]
/proc/loadavg: 8.07 8.03 6.78 5/493 31770
/proc/meminfo: memFree=19336832/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=262652 CPUtime=0.52 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 6151 0 0 0 52 0 0 0 25 0 4 0 650774598 268955648 4694 33554432000 4194304 4625164 140733678394240 18446744073709551615 4511963 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 65663 4695 345 106 0 61166 0
Current children cumulated CPU time (s) 0.52
Current children cumulated vsize (KiB) 262652

[startup+0.200753 s]
/proc/loadavg: 8.07 8.03 6.78 5/493 31770
/proc/meminfo: memFree=19336832/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=265156 CPUtime=0.66 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 6808 0 0 0 66 0 0 0 25 0 4 0 650774598 271519744 5309 33554432000 4194304 4625164 140733678394240 18446744073709551615 4512046 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 66289 5309 345 106 0 61792 0
Current children cumulated CPU time (s) 0.66
Current children cumulated vsize (KiB) 265156

[startup+0.300737 s]
/proc/loadavg: 8.07 8.03 6.78 5/493 31770
/proc/meminfo: memFree=19336832/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=266356 CPUtime=1.07 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 7648 0 0 0 106 1 0 0 25 0 4 0 650774598 272748544 6149 33554432000 4194304 4625164 140733678394240 18446744073709551615 4512032 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 66589 6149 345 106 0 62092 0
Current children cumulated CPU time (s) 1.07
Current children cumulated vsize (KiB) 266356

[startup+0.700667 s]
/proc/loadavg: 8.07 8.03 6.78 5/493 31770
/proc/meminfo: memFree=19336832/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=272408 CPUtime=2.66 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 11403 0 0 0 265 1 0 0 25 0 4 0 650774598 278945792 9336 33554432000 4194304 4625164 140733678394240 18446744073709551615 4515337 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 68102 9336 346 106 0 63605 0
Current children cumulated CPU time (s) 2.66
Current children cumulated vsize (KiB) 272408

[startup+1.50052 s]
/proc/loadavg: 8.22 8.06 6.80 9/497 31774
/proc/meminfo: memFree=19289360/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=281184 CPUtime=5.84 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 17307 0 0 0 582 2 0 0 25 0 4 0 650774598 287932416 13053 33554432000 4194304 4625164 140733678394240 18446744073709551615 4512447 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 70296 13053 348 106 0 65799 0
[pid=31770/tid=31772] ppid=31768 vsize=281184 CPUtime=1.44 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 2867 0 0 0 144 0 0 0 25 0 4 0 650774602 287932416 13053 33554432000 4194304 4625164 140733678394240 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=281184 CPUtime=1.43 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 4769 0 0 0 143 0 0 0 25 0 4 0 650774602 287932416 13054 33554432000 4194304 4625164 140733678394240 18446744073709551615 4352010 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=281184 CPUtime=1.45 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 2142 0 0 0 145 0 0 0 25 0 4 0 650774602 287932416 13054 33554432000 4194304 4625164 140733678394240 18446744073709551615 4334268 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 5.84
Current children cumulated vsize (KiB) 281184

[startup+3.10024 s]
/proc/loadavg: 8.22 8.06 6.80 9/497 31774
/proc/meminfo: memFree=19269272/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=293284 CPUtime=12.22 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 27809 0 0 0 1217 5 0 0 25 0 4 0 650774598 300322816 19571 33554432000 4194304 4625164 140733678394240 18446744073709551615 4519248 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 73321 19573 346 106 0 68824 0
[pid=31770/tid=31772] ppid=31768 vsize=293284 CPUtime=3.04 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 6723 0 0 0 303 1 0 0 25 0 4 0 650774602 300322816 19573 33554432000 4194304 4625164 140733678394240 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=293284 CPUtime=3.01 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 8148 0 0 0 299 2 0 0 25 0 4 0 650774602 300322816 19573 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351731 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=293284 CPUtime=3.05 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 3109 0 0 0 305 0 0 0 25 0 4 0 650774602 300322816 19574 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 12.22
Current children cumulated vsize (KiB) 293284

[startup+6.30067 s]
/proc/loadavg: 8.21 8.06 6.80 9/497 31774
/proc/meminfo: memFree=19223152/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=301448 CPUtime=24.97 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 57892 0 0 0 2487 10 0 0 25 0 4 0 650774598 308682752 27506 33554432000 4194304 4625164 140733678394240 18446744073709551615 4511910 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 75362 27506 347 106 0 70865 0
[pid=31770/tid=31772] ppid=31768 vsize=301448 CPUtime=6.24 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 9616 0 0 0 623 1 0 0 25 0 4 0 650774602 308682752 27506 33554432000 4194304 4625164 140733678394240 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=301448 CPUtime=6.15 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 12752 0 0 0 612 3 0 0 25 0 4 0 650774602 308682752 27506 33554432000 4194304 4625164 140733678394240 18446744073709551615 4352056 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=301448 CPUtime=6.25 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 24659 0 0 0 622 3 0 0 25 0 4 0 650774602 308682752 27506 33554432000 4194304 4625164 140733678394240 18446744073709551615 4339591 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 24.97
Current children cumulated vsize (KiB) 301448

[startup+12.7005 s]
/proc/loadavg: 8.19 8.06 6.81 9/497 31774
/proc/meminfo: memFree=19166044/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=325540 CPUtime=50.46 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 91525 0 0 0 5029 17 0 0 25 0 4 0 650774598 333352960 39805 33554432000 4194304 4625164 140733678394240 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 81385 39805 346 106 0 76888 0
[pid=31770/tid=31772] ppid=31768 vsize=325540 CPUtime=12.64 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 18380 0 0 0 1261 3 0 0 25 0 4 0 650774602 333352960 39805 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=325540 CPUtime=12.45 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 20330 0 0 0 1238 7 0 0 25 0 4 0 650774602 333352960 39805 33554432000 4194304 4625164 140733678394240 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=325540 CPUtime=12.64 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 39115 0 0 0 1260 4 0 0 25 0 4 0 650774602 333352960 39805 33554432000 4194304 4625164 140733678394240 18446744073709551615 212055639067 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 50.46
Current children cumulated vsize (KiB) 325540

[startup+25.5012 s]
/proc/loadavg: 8.15 8.05 6.83 9/497 31775
/proc/meminfo: memFree=19079732/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=348120 CPUtime=101.45 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 162153 0 0 0 10115 30 0 0 25 0 4 0 650774598 356474880 56847 33554432000 4194304 4625164 140733678394240 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 87030 56847 346 106 0 82533 0
[pid=31770/tid=31772] ppid=31768 vsize=348120 CPUtime=25.44 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 23581 0 0 0 2538 6 0 0 25 0 4 0 650774602 356474880 56847 33554432000 4194304 4625164 140733678394240 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=348120 CPUtime=25.04 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 28258 0 0 0 2496 8 0 0 25 0 4 0 650774602 356474880 56847 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=348120 CPUtime=25.45 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 94978 0 0 0 2533 12 0 0 25 0 4 0 650774602 356474880 56847 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 101.45
Current children cumulated vsize (KiB) 348120

[startup+51.1007 s]
/proc/loadavg: 8.17 8.07 6.86 9/497 31775
/proc/meminfo: memFree=18924792/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=571988 CPUtime=203.43 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 413330 0 0 0 20272 71 0 0 25 0 4 0 650774598 585715712 89624 33554432000 4194304 4625164 140733678394240 18446744073709551615 4393080 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 142997 89624 661 106 0 138500 0
[pid=31770/tid=31772] ppid=31768 vsize=571988 CPUtime=51.05 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 42064 0 0 0 5093 12 0 0 25 0 4 0 650774602 585715712 89624 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351844 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=571988 CPUtime=50.22 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 42760 0 0 0 5010 12 0 0 25 0 4 0 650774602 585715712 89624 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=571988 CPUtime=51.05 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 292048 0 0 0 5065 40 0 0 25 0 4 0 650774602 585715712 89624 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 203.43
Current children cumulated vsize (KiB) 571988

[startup+102.301 s]
/proc/loadavg: 8.19 8.08 6.93 9/497 31777
/proc/meminfo: memFree=18662628/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=794108 CPUtime=407.46 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 720905 0 0 0 40616 130 0 0 25 0 4 0 650774598 813166592 146390 33554432000 4194304 4625164 140733678394240 18446744073709551615 4393051 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 198527 146390 383 106 0 194030 0
[pid=31770/tid=31772] ppid=31768 vsize=794108 CPUtime=102.26 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 64598 0 0 0 10202 24 0 0 25 0 4 0 650774602 813166592 146390 33554432000 4194304 4625164 140733678394240 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=794108 CPUtime=100.6 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 62464 0 0 0 10039 21 0 0 25 0 4 0 650774602 813166592 146390 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351786 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=794108 CPUtime=102.26 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 482519 0 0 0 10159 67 0 0 25 0 4 0 650774602 813166592 146390 33554432000 4194304 4625164 140733678394240 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 407.46
Current children cumulated vsize (KiB) 794108

[startup+162.301 s]
/proc/loadavg: 8.21 8.11 7.01 9/497 31778
/proc/meminfo: memFree=18366588/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=997340 CPUtime=646.42 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 908441 0 0 0 64466 176 0 0 25 0 4 0 650774598 1021276160 205423 33554432000 4194304 4625164 140733678394240 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 249335 205423 534 106 0 244838 0
[pid=31770/tid=31772] ppid=31768 vsize=997340 CPUtime=162.24 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 74313 0 0 0 16190 34 0 0 25 0 4 0 650774602 1021276160 205423 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=997340 CPUtime=159.61 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 91754 0 0 0 15926 35 0 0 25 0 4 0 650774602 1021276160 205423 33554432000 4194304 4625164 140733678394240 18446744073709551615 4352014 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=997340 CPUtime=162.24 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 574613 0 0 0 16143 81 0 0 25 0 4 0 650774602 1021276160 205423 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 646.42
Current children cumulated vsize (KiB) 997340

[startup+222.319 s]
/proc/loadavg: 8.24 8.14 7.09 9/497 31780
/proc/meminfo: memFree=18135148/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=1287776 CPUtime=885.5 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 1048076 0 0 0 88337 213 0 0 25 0 4 0 650774598 1318682624 252339 33554432000 4194304 4625164 140733678394240 18446744073709551615 4392971 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 321944 252339 533 106 0 317447 0
[pid=31770/tid=31772] ppid=31768 vsize=1287776 CPUtime=222.26 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 101973 0 0 0 22179 47 0 0 25 0 4 0 650774602 1318682624 252339 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=1287776 CPUtime=218.64 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 125196 0 0 0 21816 48 0 0 25 0 4 0 650774602 1318682624 252339 33554432000 4194304 4625164 140733678394240 18446744073709551615 4352072 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=1287776 CPUtime=222.27 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 579114 0 0 0 22144 83 0 0 25 0 4 0 650774602 1318682624 252339 33554432000 4194304 4625164 140733678394240 18446744073709551615 4559888 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 885.5
Current children cumulated vsize (KiB) 1287776

[startup+282.3 s]

################
# More data... #
################

[pid=31770] ppid=31768 vsize=2135088 CPUtime=2797.61 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 1804203 0 0 0 279372 389 0 0 25 0 4 0 650774598 2186330112 437018 33554432000 4194304 4625164 140733678394240 18446744073709551615 4393061 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 533772 437018 346 106 0 529275 0
[pid=31770/tid=31772] ppid=31768 vsize=2135088 CPUtime=702.25 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 120579 0 0 0 70160 65 0 0 25 0 4 0 650774602 2186330112 437018 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331104 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=2135088 CPUtime=690.77 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 202623 0 0 0 68979 98 0 0 25 0 4 0 650774602 2186330112 437018 33554432000 4194304 4625164 140733678394240 18446744073709551615 4330603 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=2135088 CPUtime=702.27 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 636669 0 0 0 70132 95 0 0 25 0 4 0 650774602 2186330112 437018 33554432000 4194304 4625164 140733678394240 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2797.61
Current children cumulated vsize (KiB) 2135088

[startup+762.301 s]
/proc/loadavg: 8.19 8.16 7.54 9/497 31794
/proc/meminfo: memFree=16907844/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=2279332 CPUtime=3036.69 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 1892498 0 0 0 303264 405 0 0 25 0 4 0 650774598 2334035968 448054 33554432000 4194304 4625164 140733678394240 18446744073709551615 4393110 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 569833 448054 346 106 0 565336 0
[pid=31770/tid=31772] ppid=31768 vsize=2279332 CPUtime=762.27 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 120759 0 0 0 76162 65 0 0 25 0 4 0 650774602 2334035968 448054 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=2279332 CPUtime=749.8 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 205657 0 0 0 74882 98 0 0 25 0 4 0 650774602 2334035968 448054 33554432000 4194304 4625164 140733678394240 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=2279332 CPUtime=762.28 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 640267 0 0 0 76133 95 0 0 25 0 4 0 650774602 2334035968 448054 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3036.69
Current children cumulated vsize (KiB) 2279332

[startup+822.301 s]
/proc/loadavg: 8.18 8.17 7.58 9/497 31796
/proc/meminfo: memFree=16804016/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=2291332 CPUtime=3275.65 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 1993078 0 0 0 327139 426 0 0 25 0 4 0 650774598 2346323968 454767 33554432000 4194304 4625164 140733678394240 18446744073709551615 4393080 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 572833 454767 346 106 0 568336 0
[pid=31770/tid=31772] ppid=31768 vsize=2291332 CPUtime=822.26 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 121231 0 0 0 82160 66 0 0 25 0 4 0 650774602 2346323968 454767 33554432000 4194304 4625164 140733678394240 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=2291332 CPUtime=808.81 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 205688 0 0 0 80783 98 0 0 25 0 4 0 650774602 2346323968 454767 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=2291332 CPUtime=822.27 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 643921 0 0 0 82130 97 0 0 25 0 4 0 650774602 2346323968 454767 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351987 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3275.65
Current children cumulated vsize (KiB) 2291332

[startup+882.319 s]
/proc/loadavg: 8.14 8.16 7.62 9/497 31797
/proc/meminfo: memFree=16742220/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=2302188 CPUtime=3514.74 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 2074885 0 0 0 351035 439 0 0 25 0 4 0 650774598 2357440512 460954 33554432000 4194304 4625164 140733678394240 18446744073709551615 4393110 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 575547 460954 346 106 0 571050 0
[pid=31770/tid=31772] ppid=31768 vsize=2302188 CPUtime=882.28 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 121510 0 0 0 88162 66 0 0 25 0 4 0 650774602 2357440512 460954 33554432000 4194304 4625164 140733678394240 18446744073709551615 4330610 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=2302188 CPUtime=867.84 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 205793 0 0 0 86686 98 0 0 25 0 4 0 650774602 2357440512 460954 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=2302188 CPUtime=882.28 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 646907 0 0 0 88131 97 0 0 25 0 4 0 650774602 2357440512 460954 33554432000 4194304 4625164 140733678394240 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3514.74
Current children cumulated vsize (KiB) 2302188

[startup+942.3 s]
/proc/loadavg: 8.09 8.15 7.64 9/497 31799
/proc/meminfo: memFree=16687736/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=2313524 CPUtime=3753.7 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 2163713 0 0 0 374914 456 0 0 25 0 4 0 650774598 2369048576 467324 33554432000 4194304 4625164 140733678394240 18446744073709551615 4508979 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 578381 467324 346 106 0 573884 0
[pid=31770/tid=31772] ppid=31768 vsize=2313524 CPUtime=942.26 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 121611 0 0 0 94160 66 0 0 25 0 4 0 650774602 2369048576 467324 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=2313524 CPUtime=926.85 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 205818 0 0 0 92587 98 0 0 25 0 4 0 650774602 2369048576 467324 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=2313524 CPUtime=942.27 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 650215 0 0 0 94129 98 0 0 25 0 4 0 650774602 2369048576 467324 33554432000 4194304 4625164 140733678394240 18446744073709551615 4547614 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3753.7
Current children cumulated vsize (KiB) 2313524

[startup+1002.3 s]
/proc/loadavg: 6.12 7.63 7.50 5/488 31819
/proc/meminfo: memFree=17824900/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=2324908 CPUtime=3992.71 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 2268330 0 0 0 398794 477 0 0 25 0 4 0 650774598 2380705792 473912 33554432000 4194304 4625164 140733678394240 18446744073709551615 4393068 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 581227 473912 346 106 0 576730 0
[pid=31770/tid=31772] ppid=31768 vsize=2324908 CPUtime=1002.26 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 121711 0 0 0 100160 66 0 0 25 0 4 0 650774602 2380705792 473912 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=2324908 CPUtime=985.86 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 205845 0 0 0 98488 98 0 0 25 0 4 0 650774602 2380705792 473912 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=2324908 CPUtime=1002.27 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 653575 0 0 0 100129 98 0 0 25 0 4 0 650774602 2380705792 473912 33554432000 4194304 4625164 140733678394240 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3992.71
Current children cumulated vsize (KiB) 2324908

[startup+1062.3 s]
/proc/loadavg: 4.78 6.97 7.28 5/488 31820
/proc/meminfo: memFree=17786428/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=2387772 CPUtime=4231.74 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 2369231 0 0 0 422679 495 0 0 25 0 4 0 650774598 2445078528 483379 33554432000 4194304 4625164 140733678394240 18446744073709551615 4393080 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 596943 483379 346 106 0 592446 0
[pid=31770/tid=31772] ppid=31768 vsize=2387772 CPUtime=1062.26 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 148759 0 0 0 106156 70 0 0 25 0 4 0 650774602 2445078528 483379 33554432000 4194304 4625164 140733678394240 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=2387772 CPUtime=1044.89 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 205905 0 0 0 104391 98 0 0 25 0 4 0 650774602 2445078528 483379 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=2387772 CPUtime=1062.27 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 657121 0 0 0 106129 98 0 0 25 0 4 0 650774602 2445078528 483379 33554432000 4194304 4625164 140733678394240 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4231.74
Current children cumulated vsize (KiB) 2387772

[startup+1122.3 s]
/proc/loadavg: 4.36 6.44 7.07 5/488 31825
/proc/meminfo: memFree=17756248/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=2397960 CPUtime=4470.75 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 2410534 0 0 0 446571 504 0 0 25 0 4 0 650774598 2455511040 491018 33554432000 4194304 4625164 140733678394240 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 599490 491018 346 106 0 594993 0
[pid=31770/tid=31772] ppid=31768 vsize=2397960 CPUtime=1122.26 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 148872 0 0 0 112156 70 0 0 25 0 4 0 650774602 2455511040 491018 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=2397960 CPUtime=1103.9 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 205929 0 0 0 110292 98 0 0 25 0 4 0 650774602 2455511040 491018 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=2397960 CPUtime=1122.27 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 660272 0 0 0 112128 99 0 0 25 0 4 0 650774602 2455511040 491018 33554432000 4194304 4625164 140733678394240 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4470.75
Current children cumulated vsize (KiB) 2397960

[startup+1182.3 s]
/proc/loadavg: 4.13 5.99 6.88 5/488 31826
/proc/meminfo: memFree=17729500/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=2409284 CPUtime=4709.79 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 2473299 0 0 0 470462 517 0 0 25 0 4 0 650774598 2467106816 498172 33554432000 4194304 4625164 140733678394240 18446744073709551615 4393051 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 602321 498172 346 106 0 597824 0
[pid=31770/tid=31772] ppid=31768 vsize=2409284 CPUtime=1182.26 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 149428 0 0 0 118156 70 0 0 25 0 4 0 650774602 2467106816 498172 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=2409284 CPUtime=1162.94 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 205950 0 0 0 116195 99 0 0 25 0 4 0 650774602 2467106816 498172 33554432000 4194304 4625164 140733678394240 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=2409284 CPUtime=1182.27 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 663308 0 0 0 118127 100 0 0 25 0 4 0 650774602 2467106816 498172 33554432000 4194304 4625164 140733678394240 18446744073709551615 4334485 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4709.79
Current children cumulated vsize (KiB) 2409284



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205 s]
/proc/loadavg: 4.09 5.83 6.80 5/488 31827
/proc/meminfo: memFree=17720932/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=2412708 CPUtime=4800.27 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 2517083 0 0 0 479501 526 0 0 25 0 4 0 650774598 2470612992 499811 33554432000 4194304 4625164 140733678394240 18446744073709551615 4393080 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 603177 499811 346 106 0 598680 0
[pid=31770/tid=31772] ppid=31768 vsize=2412708 CPUtime=1204.98 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 149472 0 0 0 120428 70 0 0 25 0 4 0 650774602 2470612992 499811 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=2412708 CPUtime=1185.26 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 205953 0 0 0 118427 99 0 0 25 0 4 0 650774602 2470612992 499811 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=2412708 CPUtime=1204.99 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 664282 0 0 0 120399 100 0 0 25 0 4 0 650774602 2470612992 499811 33554432000 4194304 4625164 140733678394240 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 2412708

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1205 s]
/proc/loadavg: 4.09 5.83 6.80 5/488 31827
/proc/meminfo: memFree=17720932/32951124 swapFree=51873468/67111528
[pid=31770] ppid=31768 vsize=2412708 CPUtime=4800.27 cores=1,3,5,7
/proc/31770/stat : 31770 (SArTagnan) R 31768 31770 31559 0 -1 4202496 2517083 0 0 0 479501 526 0 0 25 0 4 0 650774598 2470612992 499811 33554432000 4194304 4625164 140733678394240 18446744073709551615 4393080 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/31770/statm: 603177 499811 346 106 0 598680 0
[pid=31770/tid=31772] ppid=31768 vsize=2412708 CPUtime=1204.98 cores=1,3,5,7
/proc/31770/task/31772/stat : 31772 (SArTagnan) R 31768 31770 31559 0 -1 4202560 149472 0 0 0 120428 70 0 0 25 0 4 0 650774602 2470612992 499811 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31770/tid=31773] ppid=31768 vsize=2412708 CPUtime=1185.26 cores=1,3,5,7
/proc/31770/task/31773/stat : 31773 (SArTagnan) R 31768 31770 31559 0 -1 4202560 205953 0 0 0 118427 99 0 0 25 0 4 0 650774602 2470612992 499811 33554432000 4194304 4625164 140733678394240 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31770/tid=31774] ppid=31768 vsize=2412708 CPUtime=1204.99 cores=1,3,5,7
/proc/31770/task/31774/stat : 31774 (SArTagnan) R 31768 31770 31559 0 -1 4202560 664282 0 0 0 120399 100 0 0 25 0 4 0 650774602 2470612992 499811 33554432000 4194304 4625164 140733678394240 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 2412708

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1205.1
CPU time (s): 4800.36
CPU user time (s): 4795.03
CPU system time (s): 5.33519
CPU usage (%): 398.339
Max. virtual memory (cumulated for all children) (KiB): 2481072

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4795.03
system time used= 5.33519
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2517083
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1417
involuntary context switches= 30803

runsolver used 5.46817 second user time and 14.2818 second system time

The end

Launcher Data 

Begin job on node146 at 2011-05-17 00:06:23
IDJOB=3385840
IDBENCH=88212
IDSOLVER=1871
FILE ID=node146/3385840-1305583583
RUNJOBID= node146-1305582130-31578
PBS_JOBID= 13350290
Free space on /tmp= 73204 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m37-q505.75-n35-p15-s48276711.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385840-1305583583/watcher-3385840-1305583583 -o /tmp/evaluation-result-3385840-1305583583/solver-3385840-1305583583 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3385840-1305583583.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1bac7b96ea6eb843b1de7405a721cd87
RANDOM SEED=1310946099

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      19337112 kB
Buffers:        222224 kB
Cached:        1425240 kB
SwapCached:      56140 kB
Active:       10492752 kB
Inactive:      2845208 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      19337112 kB
SwapTotal:    67111528 kB
SwapFree:     51873468 kB
Dirty:            2364 kB
Writeback:           0 kB
AnonPages:    11641288 kB
Mapped:          26492 kB
Slab:           114972 kB
PageTables:      92468 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 140271556 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73204 MiB
End job on node146 at 2011-05-17 00:26:30