Trace number 3439251

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.9 5011.11

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_70_26_40_6-dir.shuffled-as.sat03-163.cnf
MD5SUMc06088cac8521bfe72bbb4c9ad8d39fb
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark3288.15
Satisfiable
(Un)Satisfiability was proved
Number of variables780
Number of clauses66810
Sum of the clauses size134348
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 266784
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 526

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3439251-1306960547/watcher-3439251-1306960547 -o /tmp/evaluation-result-3439251-1306960547/solver-3439251-1306960547 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3439251-1306960547.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.29 3.08 1.16 3/482 7037
/proc/meminfo: memFree=27545268/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=20488 CPUtime=0 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 613 0 0 0 0 0 0 0 25 0 1 0 807653196 20979712 479 33554432000 4194304 4625164 140733961623584 18446744073709551615 210204647168 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/7037/statm: 5122 480 296 106 0 625 0

[startup+0.097663 s]
/proc/loadavg: 7.29 3.08 1.16 3/482 7037
/proc/meminfo: memFree=27545268/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=443508 CPUtime=0.58 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 12021 0 0 0 57 1 0 0 25 0 8 0 807653196 454152192 8265 33554432000 4194304 4625164 140733961623584 18446744073709551615 210204690567 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 110877 8265 396 106 0 106380 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 443508

[startup+0.136511 s]
/proc/loadavg: 7.29 3.08 1.16 3/482 7037
/proc/meminfo: memFree=27545268/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=527544 CPUtime=1.11 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 23208 0 0 0 108 3 0 0 25 0 8 0 807653196 540205056 15188 33554432000 4194304 4625164 140733961623584 18446744073709551615 210204690567 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 131886 14676 586 106 0 127389 0
Current children cumulated CPU time (s) 1.11
Current children cumulated vsize (KiB) 527544

[startup+1.34358 s]
/proc/loadavg: 7.29 3.08 1.16 3/482 7037
/proc/meminfo: memFree=27545268/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=535056 CPUtime=10.49 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 50770 0 0 0 1039 10 0 0 18 0 8 0 807653196 547897344 31461 33554432000 4194304 4625164 140733961623584 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 133764 31461 536 106 0 129267 0
Current children cumulated CPU time (s) 10.49
Current children cumulated vsize (KiB) 535056

[startup+1.44356 s]
/proc/loadavg: 7.29 3.08 1.16 3/482 7037
/proc/meminfo: memFree=27545268/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=535056 CPUtime=11.28 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 52236 0 0 0 1118 10 0 0 19 0 8 0 807653196 547897344 31972 33554432000 4194304 4625164 140733961623584 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 133764 31972 536 106 0 129267 0
Current children cumulated CPU time (s) 11.28
Current children cumulated vsize (KiB) 535056

[startup+1.54354 s]
/proc/loadavg: 7.29 3.08 1.16 3/482 7037
/proc/meminfo: memFree=27545268/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=535056 CPUtime=12.08 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 52856 0 0 0 1198 10 0 0 19 0 8 0 807653196 547897344 32508 33554432000 4194304 4625164 140733961623584 18446744073709551615 4393124 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 133764 32508 534 106 0 129267 0
Current children cumulated CPU time (s) 12.08
Current children cumulated vsize (KiB) 535056

[startup+3.10128 s]
/proc/loadavg: 7.29 3.08 1.16 9/490 7045
/proc/meminfo: memFree=27384596/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=542460 CPUtime=24.58 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 66396 0 0 0 2446 12 0 0 22 0 8 0 807653196 555479040 40133 33554432000 4194304 4625164 140733961623584 18446744073709551615 4519328 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 135615 40133 598 106 0 131118 0
[pid=7037/tid=7039] ppid=7035 vsize=542460 CPUtime=3.08 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 6891 0 0 0 307 1 0 0 25 0 8 0 807653198 555479040 40148 33554432000 4194304 4625164 140733961623584 18446744073709551615 4339639 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=542460 CPUtime=3.09 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 4596 0 0 0 308 1 0 0 25 0 8 0 807653198 555479040 40148 33554432000 4194304 4625164 140733961623584 18446744073709551615 210204752415 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=542460 CPUtime=3.03 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 5515 0 0 0 302 1 0 0 22 0 8 0 807653198 555479040 40148 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=7037/tid=7042] ppid=7035 vsize=542460 CPUtime=3.09 cores=0-7
/proc/7037/task/7042/stat : 7042 (SArTagnan) R 7035 7037 6973 0 -1 4202560 5425 0 0 0 308 1 0 0 25 0 8 0 807653198 555479040 40148 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=7037/tid=7043] ppid=7035 vsize=542460 CPUtime=3.07 cores=0-7
/proc/7037/task/7043/stat : 7043 (SArTagnan) R 7035 7037 6973 0 -1 4202560 5562 0 0 0 307 0 0 0 22 0 8 0 807653198 555479040 40148 33554432000 4194304 4625164 140733961623584 18446744073709551615 4329456 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=7037/tid=7044] ppid=7035 vsize=542460 CPUtime=3.08 cores=0-7
/proc/7037/task/7044/stat : 7044 (SArTagnan) R 7035 7037 6973 0 -1 4202560 5588 0 0 0 307 1 0 0 22 0 8 0 807653198 555479040 40148 33554432000 4194304 4625164 140733961623584 18446744073709551615 4339639 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=7037/tid=7045] ppid=7035 vsize=542460 CPUtime=3.07 cores=0-7
/proc/7037/task/7045/stat : 7045 (SArTagnan) R 7035 7037 6973 0 -1 4202560 4360 0 0 0 307 0 0 0 25 0 8 0 807653198 555479040 40148 33554432000 4194304 4625164 140733961623584 18446744073709551615 4352157 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.58
Current children cumulated vsize (KiB) 542460

[startup+6.30075 s]
/proc/loadavg: 7.35 3.16 1.19 9/490 7045
/proc/meminfo: memFree=27356200/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=555196 CPUtime=50.06 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 104070 0 0 0 4987 19 0 0 25 0 8 0 807653196 568520704 54187 33554432000 4194304 4625164 140733961623584 18446744073709551615 4393055 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 138799 54187 752 106 0 134302 0
[pid=7037/tid=7039] ppid=7035 vsize=555196 CPUtime=6.26 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 10163 0 0 0 625 1 0 0 25 0 8 0 807653198 568520704 54187 33554432000 4194304 4625164 140733961623584 18446744073709551615 4352002 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=555196 CPUtime=6.26 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 7211 0 0 0 625 1 0 0 22 0 8 0 807653198 568520704 54187 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=555196 CPUtime=6.18 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 19228 0 0 0 614 4 0 0 25 0 8 0 807653198 568520704 54187 33554432000 4194304 4625164 140733961623584 18446744073709551615 4546803 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=7037/tid=7042] ppid=7035 vsize=555196 CPUtime=6.26 cores=0-7
/proc/7037/task/7042/stat : 7042 (SArTagnan) R 7035 7037 6973 0 -1 4202560 7705 0 0 0 625 1 0 0 25 0 8 0 807653198 568520704 54187 33554432000 4194304 4625164 140733961623584 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=7037/tid=7043] ppid=7035 vsize=555196 CPUtime=6.25 cores=0-7
/proc/7037/task/7043/stat : 7043 (SArTagnan) R 7035 7037 6973 0 -1 4202560 8390 0 0 0 624 1 0 0 25 0 8 0 807653198 568520704 54187 33554432000 4194304 4625164 140733961623584 18446744073709551615 4319706 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=7037/tid=7044] ppid=7035 vsize=555196 CPUtime=6.25 cores=0-7
/proc/7037/task/7044/stat : 7044 (SArTagnan) R 7035 7037 6973 0 -1 4202560 8543 0 0 0 624 1 0 0 25 0 8 0 807653198 568520704 54187 33554432000 4194304 4625164 140733961623584 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=7037/tid=7045] ppid=7035 vsize=555196 CPUtime=6.26 cores=0-7
/proc/7037/task/7045/stat : 7045 (SArTagnan) R 7035 7037 6973 0 -1 4202560 6445 0 0 0 625 1 0 0 25 0 8 0 807653198 568520704 54187 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.06
Current children cumulated vsize (KiB) 555196

[startup+12.7007 s]
/proc/loadavg: 7.40 3.24 1.23 9/490 7045
/proc/meminfo: memFree=27252960/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=568636 CPUtime=101.15 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 144179 0 0 0 10089 26 0 0 25 0 8 0 807653196 582283264 75784 33554432000 4194304 4625164 140733961623584 18446744073709551615 4512017 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 142159 75784 979 106 0 137662 0
[pid=7037/tid=7039] ppid=7035 vsize=568636 CPUtime=12.67 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 13161 0 0 0 1265 2 0 0 25 0 8 0 807653198 582283264 75784 33554432000 4194304 4625164 140733961623584 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=568636 CPUtime=12.67 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 10592 0 0 0 1265 2 0 0 22 0 8 0 807653198 582283264 75784 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=568636 CPUtime=12.47 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 38782 0 0 0 1241 6 0 0 25 0 8 0 807653198 582283264 75784 33554432000 4194304 4625164 140733961623584 18446744073709551615 4547221 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=7037/tid=7042] ppid=7035 vsize=568636 CPUtime=12.66 cores=0-7
/proc/7037/task/7042/stat : 7042 (SArTagnan) R 7035 7037 6973 0 -1 4202560 8567 0 0 0 1265 1 0 0 25 0 8 0 807653198 582283264 75784 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=7037/tid=7043] ppid=7035 vsize=568636 CPUtime=12.65 cores=0-7
/proc/7037/task/7043/stat : 7043 (SArTagnan) R 7035 7037 6973 0 -1 4202560 13425 0 0 0 1263 2 0 0 25 0 8 0 807653198 582283264 75784 33554432000 4194304 4625164 140733961623584 18446744073709551615 4348900 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=7037/tid=7044] ppid=7035 vsize=568636 CPUtime=12.66 cores=0-7
/proc/7037/task/7044/stat : 7044 (SArTagnan) R 7035 7037 6973 0 -1 4202560 12168 0 0 0 1264 2 0 0 25 0 8 0 807653198 582283264 75784 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=7037/tid=7045] ppid=7035 vsize=568636 CPUtime=12.66 cores=0-7
/proc/7037/task/7045/stat : 7045 (SArTagnan) R 7035 7037 6973 0 -1 4202560 7167 0 0 0 1265 1 0 0 25 0 8 0 807653198 582283264 75784 33554432000 4194304 4625164 140733961623584 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.15
Current children cumulated vsize (KiB) 568636

[startup+25.5005 s]
/proc/loadavg: 7.53 3.47 1.34 9/490 7045
/proc/meminfo: memFree=27096160/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=816552 CPUtime=203.33 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 226590 0 0 0 20294 39 0 0 25 0 8 0 807653196 836149248 114293 33554432000 4194304 4625164 140733961623584 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 204138 114293 1653 106 0 199641 0
[pid=7037/tid=7039] ppid=7035 vsize=816552 CPUtime=25.46 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 20044 0 0 0 2543 3 0 0 25 0 8 0 807653198 836149248 114293 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=816552 CPUtime=25.47 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 15393 0 0 0 2544 3 0 0 25 0 8 0 807653198 836149248 114293 33554432000 4194304 4625164 140733961623584 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=816552 CPUtime=25.06 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 76640 0 0 0 2494 12 0 0 25 0 8 0 807653198 836149248 114293 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=7037/tid=7042] ppid=7035 vsize=816552 CPUtime=25.47 cores=0-7
/proc/7037/task/7042/stat : 7042 (SArTagnan) R 7035 7037 6973 0 -1 4202560 12006 0 0 0 2545 2 0 0 25 0 8 0 807653198 836149248 114293 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=7037/tid=7043] ppid=7035 vsize=816552 CPUtime=25.44 cores=0-7
/proc/7037/task/7043/stat : 7043 (SArTagnan) R 7035 7037 6973 0 -1 4202560 19993 0 0 0 2542 2 0 0 25 0 8 0 807653198 836149248 114293 33554432000 4194304 4625164 140733961623584 18446744073709551615 4348821 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=7037/tid=7044] ppid=7035 vsize=816552 CPUtime=25.45 cores=0-7
/proc/7037/task/7044/stat : 7044 (SArTagnan) R 7035 7037 6973 0 -1 4202560 17991 0 0 0 2542 3 0 0 25 0 8 0 807653198 836149248 114293 33554432000 4194304 4625164 140733961623584 18446744073709551615 210204303317 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=7037/tid=7045] ppid=7035 vsize=816552 CPUtime=25.45 cores=0-7
/proc/7037/task/7045/stat : 7045 (SArTagnan) R 7035 7037 6973 0 -1 4202560 11285 0 0 0 2543 2 0 0 25 0 8 0 807653198 836149248 114293 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 203.33
Current children cumulated vsize (KiB) 816552

[startup+51.1012 s]
/proc/loadavg: 7.69 3.84 1.51 9/490 7046
/proc/meminfo: memFree=26926192/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=1015840 CPUtime=407.71 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 352568 0 0 0 40710 61 0 0 25 0 8 0 807653196 1040220160 156939 33554432000 4194304 4625164 140733961623584 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 253960 156939 1968 106 0 249463 0
[pid=7037/tid=7039] ppid=7035 vsize=1015840 CPUtime=51.07 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 32135 0 0 0 5102 5 0 0 25 0 8 0 807653198 1040220160 156939 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=1015840 CPUtime=51.06 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 16399 0 0 0 5103 3 0 0 25 0 8 0 807653198 1040220160 156939 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=1015840 CPUtime=50.24 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 130909 0 0 0 5002 22 0 0 25 0 8 0 807653198 1040220160 156939 33554432000 4194304 4625164 140733961623584 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=7037/tid=7042] ppid=7035 vsize=1015840 CPUtime=51.06 cores=0-7
/proc/7037/task/7042/stat : 7042 (SArTagnan) R 7035 7037 6973 0 -1 4202560 13197 0 0 0 5104 2 0 0 25 0 8 0 807653198 1040220160 156939 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=7037/tid=7043] ppid=7035 vsize=1015840 CPUtime=51.03 cores=0-7
/proc/7037/task/7043/stat : 7043 (SArTagnan) R 7035 7037 6973 0 -1 4202560 31049 0 0 0 5098 5 0 0 25 0 8 0 807653198 1040220160 156939 33554432000 4194304 4625164 140733961623584 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=7037/tid=7044] ppid=7035 vsize=1015840 CPUtime=51.06 cores=0-7
/proc/7037/task/7044/stat : 7044 (SArTagnan) R 7035 7037 6973 0 -1 4202560 35688 0 0 0 5101 5 0 0 25 0 8 0 807653198 1040220160 156939 33554432000 4194304 4625164 140733961623584 18446744073709551615 4339660 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=7037/tid=7045] ppid=7035 vsize=1015840 CPUtime=51.06 cores=0-7
/proc/7037/task/7045/stat : 7045 (SArTagnan) R 7035 7037 6973 0 -1 4202560 16518 0 0 0 5103 3 0 0 25 0 8 0 807653198 1040220160 156939 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 407.71
Current children cumulated vsize (KiB) 1015840

[startup+102.301 s]
/proc/loadavg: 7.86 4.48 1.85 9/490 7047
/proc/meminfo: memFree=26567656/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=1409024 CPUtime=816.43 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 661529 0 0 0 81535 108 0 0 25 0 8 0 807653196 1442840576 245406 33554432000 4194304 4625164 140733961623584 18446744073709551615 4393101 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 352256 245406 890 106 0 347759 0
[pid=7037/tid=7039] ppid=7035 vsize=1409024 CPUtime=102.26 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 52083 0 0 0 10218 8 0 0 25 0 8 0 807653198 1442840576 245406 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351827 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=1409024 CPUtime=102.26 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 25966 0 0 0 10221 5 0 0 25 0 8 0 807653198 1442840576 245406 33554432000 4194304 4625164 140733961623584 18446744073709551615 4331255 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=1409024 CPUtime=100.59 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 261467 0 0 0 10018 41 0 0 25 0 8 0 807653198 1442840576 245406 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0

################
# More data... #
################

[pid=7037] ppid=7035 vsize=10553452 CPUtime=37697.9 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 38642670 0 0 0 3763396 6395 0 0 25 0 8 0 807653196 10806734848 2159180 33554432000 4194304 4625164 140733961623584 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/7037/statm: 2638363 2159180 353 106 0 2633866 0
[pid=7037/tid=7039] ppid=7035 vsize=10553452 CPUtime=4722.29 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 277762 0 0 0 472179 50 0 0 25 0 8 0 807653198 10806734848 2159180 33554432000 4194304 4625164 140733961623584 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=10553452 CPUtime=4722.21 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 259921 0 0 0 472122 99 0 0 25 0 8 0 807653198 10806734848 2159183 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=10553452 CPUtime=4689.04 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 32442132 0 0 0 463691 5213 0 0 25 0 8 0 807653198 10806734848 2159183 33554432000 4194304 4625164 140733961623584 18446744073709551615 4579396 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=7037/tid=7042] ppid=7035 vsize=10553452 CPUtime=4722.17 cores=0-7
/proc/7037/task/7042/stat : 7042 (SArTagnan) R 7035 7037 6973 0 -1 4202560 262724 0 0 0 472117 100 0 0 25 0 8 0 807653198 10806734848 2159183 33554432000 4194304 4625164 140733961623584 18446744073709551615 4352285 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=7037/tid=7043] ppid=7035 vsize=10553452 CPUtime=4721.09 cores=0-7
/proc/7037/task/7043/stat : 7043 (SArTagnan) R 7035 7037 6973 0 -1 4202560 256473 0 0 0 472058 51 0 0 25 0 8 0 807653198 10806734848 2159186 33554432000 4194304 4625164 140733961623584 18446744073709551615 4460412 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=7037/tid=7044] ppid=7035 vsize=10553452 CPUtime=4721.61 cores=0-7
/proc/7037/task/7044/stat : 7044 (SArTagnan) R 7035 7037 6973 0 -1 4202560 314654 0 0 0 472104 57 0 0 25 0 8 0 807653198 10806734848 2159186 33554432000 4194304 4625164 140733961623584 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=7037/tid=7045] ppid=7035 vsize=10553452 CPUtime=4722.08 cores=0-7
/proc/7037/task/7045/stat : 7045 (SArTagnan) R 7035 7037 6973 0 -1 4202560 224783 0 0 0 472138 70 0 0 25 0 8 0 807653198 10806734848 2159186 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37697.9
Current children cumulated vsize (KiB) 10553452

[startup+4782.3 s]
/proc/loadavg: 8.13 8.10 8.01 9/490 7184
/proc/meminfo: memFree=18819404/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=10618988 CPUtime=38176.9 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 39215584 0 0 0 3811203 6489 0 0 25 0 8 0 807653196 10873843712 2172221 33554432000 4194304 4625164 140733961623584 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/7037/statm: 2654747 2172221 379 106 0 2650250 0
[pid=7037/tid=7039] ppid=7035 vsize=10618988 CPUtime=4782.29 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 277952 0 0 0 478179 50 0 0 25 0 8 0 807653198 10873843712 2172221 33554432000 4194304 4625164 140733961623584 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=10618988 CPUtime=4782.21 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 260558 0 0 0 478121 100 0 0 25 0 8 0 807653198 10873843712 2172221 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=10618988 CPUtime=4749.05 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 32970013 0 0 0 469608 5297 0 0 25 0 8 0 807653198 10873843712 2172221 33554432000 4194304 4625164 140733961623584 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=7037/tid=7042] ppid=7035 vsize=10618988 CPUtime=4782.17 cores=0-7
/proc/7037/task/7042/stat : 7042 (SArTagnan) R 7035 7037 6973 0 -1 4202560 263250 0 0 0 478117 100 0 0 25 0 8 0 807653198 10873843712 2172221 33554432000 4194304 4625164 140733961623584 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=7037/tid=7043] ppid=7035 vsize=10618988 CPUtime=4781.06 cores=0-7
/proc/7037/task/7043/stat : 7043 (SArTagnan) R 7035 7037 6973 0 -1 4202560 256822 0 0 0 478055 51 0 0 25 0 8 0 807653198 10873843712 2172221 33554432000 4194304 4625164 140733961623584 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=7037/tid=7044] ppid=7035 vsize=10618988 CPUtime=4781.61 cores=0-7
/proc/7037/task/7044/stat : 7044 (SArTagnan) R 7035 7037 6973 0 -1 4202560 314962 0 0 0 478104 57 0 0 25 0 8 0 807653198 10873843712 2172221 33554432000 4194304 4625164 140733961623584 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=7037/tid=7045] ppid=7035 vsize=10618988 CPUtime=4782.08 cores=0-7
/proc/7037/task/7045/stat : 7045 (SArTagnan) R 7035 7037 6973 0 -1 4202560 225455 0 0 0 478138 70 0 0 25 0 8 0 807653198 10873843712 2172221 33554432000 4194304 4625164 140733961623584 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38176.9
Current children cumulated vsize (KiB) 10618988

[startup+4842.3 s]
/proc/loadavg: 8.08 8.10 8.01 9/490 7186
/proc/meminfo: memFree=18806312/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=10618988 CPUtime=38655.9 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 39674986 0 0 0 3859031 6560 0 0 25 0 8 0 807653196 10873843712 2179985 33554432000 4194304 4625164 140733961623584 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/7037/statm: 2654747 2179985 353 106 0 2650250 0
[pid=7037/tid=7039] ppid=7035 vsize=10618988 CPUtime=4842.29 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 278076 0 0 0 484179 50 0 0 25 0 8 0 807653198 10873843712 2179985 33554432000 4194304 4625164 140733961623584 18446744073709551615 4331104 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=10618988 CPUtime=4842.22 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 262820 0 0 0 484121 101 0 0 25 0 8 0 807653198 10873843712 2179985 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=10618988 CPUtime=4809.05 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 33387042 0 0 0 475545 5360 0 0 25 0 8 0 807653198 10873843712 2179985 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=7037/tid=7042] ppid=7035 vsize=10618988 CPUtime=4842.16 cores=0-7
/proc/7037/task/7042/stat : 7042 (SArTagnan) R 7035 7037 6973 0 -1 4202560 263725 0 0 0 484116 100 0 0 25 0 8 0 807653198 10873843712 2179985 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=7037/tid=7043] ppid=7035 vsize=10618988 CPUtime=4841.05 cores=0-7
/proc/7037/task/7043/stat : 7043 (SArTagnan) R 7035 7037 6973 0 -1 4202560 256851 0 0 0 484054 51 0 0 25 0 8 0 807653198 10873843712 2179985 33554432000 4194304 4625164 140733961623584 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=7037/tid=7044] ppid=7035 vsize=10618988 CPUtime=4841.6 cores=0-7
/proc/7037/task/7044/stat : 7044 (SArTagnan) R 7035 7037 6973 0 -1 4202560 315101 0 0 0 484103 57 0 0 25 0 8 0 807653198 10873843712 2179985 33554432000 4194304 4625164 140733961623584 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=7037/tid=7045] ppid=7035 vsize=10618988 CPUtime=4842.08 cores=0-7
/proc/7037/task/7045/stat : 7045 (SArTagnan) R 7035 7037 6973 0 -1 4202560 226145 0 0 0 484137 71 0 0 25 0 8 0 807653198 10873843712 2179985 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38655.9
Current children cumulated vsize (KiB) 10618988

[startup+4902.3 s]
/proc/loadavg: 8.06 8.09 8.01 9/490 7187
/proc/meminfo: memFree=18654592/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=10684524 CPUtime=39134.9 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 40173989 0 0 0 3906853 6641 0 0 25 0 8 0 807653196 10940952576 2218844 33554432000 4194304 4625164 140733961623584 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/7037/statm: 2671131 2218844 531 106 0 2666634 0
[pid=7037/tid=7039] ppid=7035 vsize=10684524 CPUtime=4902.28 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 279514 0 0 0 490178 50 0 0 25 0 8 0 807653198 10940952576 2218844 33554432000 4194304 4625164 140733961623584 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=10684524 CPUtime=4902.22 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 263080 0 0 0 490121 101 0 0 25 0 8 0 807653198 10940952576 2218844 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=10684524 CPUtime=4869.05 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 33833020 0 0 0 481477 5428 0 0 25 0 8 0 807653198 10940952576 2218844 33554432000 4194304 4625164 140733961623584 18446744073709551615 4335407 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=7037/tid=7042] ppid=7035 vsize=10684524 CPUtime=4902.16 cores=0-7
/proc/7037/task/7042/stat : 7042 (SArTagnan) R 7035 7037 6973 0 -1 4202560 264915 0 0 0 490115 101 0 0 25 0 8 0 807653198 10940952576 2218844 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=7037/tid=7043] ppid=7035 vsize=10684524 CPUtime=4901.04 cores=0-7
/proc/7037/task/7043/stat : 7043 (SArTagnan) R 7035 7037 6973 0 -1 4202560 258222 0 0 0 490052 52 0 0 25 0 8 0 807653198 10940952576 2218844 33554432000 4194304 4625164 140733961623584 18446744073709551615 4460412 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=7037/tid=7044] ppid=7035 vsize=10684524 CPUtime=4901.61 cores=0-7
/proc/7037/task/7044/stat : 7044 (SArTagnan) R 7035 7037 6973 0 -1 4202560 316323 0 0 0 490103 58 0 0 25 0 8 0 807653198 10940952576 2218844 33554432000 4194304 4625164 140733961623584 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=7037/tid=7045] ppid=7035 vsize=10684524 CPUtime=4902.08 cores=0-7
/proc/7037/task/7045/stat : 7045 (SArTagnan) R 7035 7037 6973 0 -1 4202560 227418 0 0 0 490136 72 0 0 25 0 8 0 807653198 10940952576 2218844 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39134.9
Current children cumulated vsize (KiB) 10684524

[startup+4962.3 s]
/proc/loadavg: 8.13 8.10 8.01 9/490 7189
/proc/meminfo: memFree=18635920/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=10750060 CPUtime=39613.9 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 40716546 0 0 0 3954663 6729 0 0 25 0 8 0 807653196 11008061440 2225065 33554432000 4194304 4625164 140733961623584 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/7037/statm: 2687515 2225065 420 106 0 2683018 0
[pid=7037/tid=7039] ppid=7035 vsize=10750060 CPUtime=4962.29 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 279527 0 0 0 496178 51 0 0 25 0 8 0 807653198 11008061440 2225065 33554432000 4194304 4625164 140733961623584 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=10750060 CPUtime=4962.21 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 265053 0 0 0 496118 103 0 0 25 0 8 0 807653198 11008061440 2225065 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=10750060 CPUtime=4929.05 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 34369065 0 0 0 487396 5509 0 0 25 0 8 0 807653198 11008061440 2225065 33554432000 4194304 4625164 140733961623584 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=7037/tid=7042] ppid=7035 vsize=10750060 CPUtime=4962.17 cores=0-7
/proc/7037/task/7042/stat : 7042 (SArTagnan) R 7035 7037 6973 0 -1 4202560 266807 0 0 0 496112 105 0 0 25 0 8 0 807653198 11008061440 2225065 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=7037/tid=7043] ppid=7035 vsize=10750060 CPUtime=4961.02 cores=0-7
/proc/7037/task/7043/stat : 7043 (SArTagnan) R 7035 7037 6973 0 -1 4202560 258243 0 0 0 496050 52 0 0 25 0 8 0 807653198 11008061440 2225065 33554432000 4194304 4625164 140733961623584 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=7037/tid=7044] ppid=7035 vsize=10750060 CPUtime=4961.61 cores=0-7
/proc/7037/task/7044/stat : 7044 (SArTagnan) R 7035 7037 6973 0 -1 4202560 316532 0 0 0 496103 58 0 0 25 0 8 0 807653198 11008061440 2225065 33554432000 4194304 4625164 140733961623584 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=7037/tid=7045] ppid=7035 vsize=10750060 CPUtime=4962.08 cores=0-7
/proc/7037/task/7045/stat : 7045 (SArTagnan) R 7035 7037 6973 0 -1 4202560 228365 0 0 0 496134 74 0 0 25 0 8 0 807653198 11008061440 2225065 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39613.9
Current children cumulated vsize (KiB) 10750060



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.7 s]
/proc/loadavg: 8.06 8.08 8.01 9/490 7190
/proc/meminfo: memFree=18565572/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=10823084 CPUtime=40000.4 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4202496 41142023 0 0 0 3993242 6795 0 0 25 0 8 0 807653196 11082838016 2239974 33554432000 4194304 4625164 140733961623584 18446744073709551615 4393080 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/7037/statm: 2705771 2239974 432 106 0 2701274 0
[pid=7037/tid=7039] ppid=7035 vsize=10823084 CPUtime=5010.71 cores=0-7
/proc/7037/task/7039/stat : 7039 (SArTagnan) R 7035 7037 6973 0 -1 4202560 279620 0 0 0 501020 51 0 0 25 0 8 0 807653198 11082838016 2239974 33554432000 4194304 4625164 140733961623584 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=7037/tid=7040] ppid=7035 vsize=10823084 CPUtime=5010.63 cores=0-7
/proc/7037/task/7040/stat : 7040 (SArTagnan) R 7035 7037 6973 0 -1 4202560 265769 0 0 0 500959 104 0 0 25 0 8 0 807653198 11082838016 2239974 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=7037/tid=7041] ppid=7035 vsize=10823084 CPUtime=4977.46 cores=0-7
/proc/7037/task/7041/stat : 7041 (SArTagnan) R 7035 7037 6973 0 -1 4202560 34757870 0 0 0 492178 5568 0 0 25 0 8 0 807653198 11082838016 2239974 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=7037/tid=7042] ppid=7035 vsize=10823084 CPUtime=5010.58 cores=0-7
/proc/7037/task/7042/stat : 7042 (SArTagnan) R 7035 7037 6973 0 -1 4202560 267569 0 0 0 500953 105 0 0 25 0 8 0 807653198 11082838016 2239974 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=7037/tid=7043] ppid=7035 vsize=10823084 CPUtime=5009.43 cores=0-7
/proc/7037/task/7043/stat : 7043 (SArTagnan) R 7035 7037 6973 0 -1 4202560 258533 0 0 0 500891 52 0 0 25 0 8 0 807653198 11082838016 2239974 33554432000 4194304 4625164 140733961623584 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=7037/tid=7044] ppid=7035 vsize=10823084 CPUtime=5010.03 cores=0-7
/proc/7037/task/7044/stat : 7044 (SArTagnan) R 7035 7037 6973 0 -1 4202560 316963 0 0 0 500945 58 0 0 25 0 8 0 807653198 11082838016 2239974 33554432000 4194304 4625164 140733961623584 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=7037/tid=7045] ppid=7035 vsize=10823084 CPUtime=5010.51 cores=0-7
/proc/7037/task/7045/stat : 7045 (SArTagnan) R 7035 7037 6973 0 -1 4202560 228436 0 0 0 500976 75 0 0 25 0 8 0 807653198 11082838016 2239974 33554432000 4194304 4625164 140733961623584 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 10823084

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.82 s]
/proc/loadavg: 8.06 8.08 8.01 9/490 7190
/proc/meminfo: memFree=18565572/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=0 CPUtime=40000.6 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4203524 41142023 0 0 0 3993254 6805 0 0 25 0 1 0 807653196 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 0

[startup+5011 s]
/proc/loadavg: 8.06 8.08 8.01 9/490 7190
/proc/meminfo: memFree=18565572/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=0 CPUtime=40000.8 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4203524 41142023 0 0 0 3993254 6822 0 0 25 0 1 0 807653196 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 0

[startup+5011.1 s]
/proc/loadavg: 8.06 8.08 8.01 9/490 7190
/proc/meminfo: memFree=18565572/32951124 swapFree=63649284/67111528
[pid=7037] ppid=7035 vsize=0 CPUtime=40000.9 cores=0-7
/proc/7037/stat : 7037 (SArTagnan) R 7035 7037 6973 0 -1 4203524 41142023 0 0 0 3993254 6832 0 0 25 0 1 0 807653196 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/7037/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5011.11
CPU time (s): 40000.9
CPU user time (s): 39932.5
CPU system time (s): 68.3396
CPU usage (%): 798.243
Max. virtual memory (cumulated for all children) (KiB): 11103324

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39932.5
system time used= 68.3396
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 41142023
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 29849
involuntary context switches= 176699

runsolver used 21.9917 second user time and 60.3948 second system time

The end

Launcher Data 

Begin job on node130 at 2011-06-01 22:35:47
IDJOB=3439251
IDBENCH=83903
IDSOLVER=1936
FILE ID=node130/3439251-1306960547
RUNJOBID= node130-1306960406-6987
PBS_JOBID= 13497621
Free space on /tmp= 73792 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_70_26_40_6-dir.shuffled-as.sat03-163.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3439251-1306960547/watcher-3439251-1306960547 -o /tmp/evaluation-result-3439251-1306960547/solver-3439251-1306960547 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3439251-1306960547.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= c06088cac8521bfe72bbb4c9ad8d39fb
RANDOM SEED=1346475905

node130.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      27545548 kB
Buffers:        158216 kB
Cached:         191404 kB
SwapCached:       9700 kB
Active:        4921372 kB
Inactive:       208128 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      27545548 kB
SwapTotal:    67111528 kB
SwapFree:     63649284 kB
Dirty:            1176 kB
Writeback:           0 kB
AnonPages:     4778436 kB
Mapped:          30276 kB
Slab:           151284 kB
PageTables:      53536 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 145070508 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73792 MiB
End job on node130 at 2011-06-01 23:59:20