Trace number 3317051

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 2779.03 720.518

General information on the benchmark

Namerandom/large/
unif-k5-r20-v1500-c30000-S976428817-077-UNKNOWN.cnf
MD5SUM4d2cfd51677809bb15d90c916352e4e2
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark353.72
Satisfiable
(Un)Satisfiability was proved
Number of variables1500
Number of clauses30000
Sum of the clauses size150000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 530000
Number of clauses of size over 50

Solver Data 

0.58/21.49	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.58/21.49	c INFO: solving HOME/instance-3317051-1304267280.cnf
0.58/21.49	c INFO: computing task features
0.70/21.64	c INFO: computed variable-clause graph statistics
0.70/21.64	c INFO: computed clause balance statistics
1.50/22.40	c INFO: computed variable balance statistics
1.90/22.80	c INFO: computed small-clause counts
1.90/22.81	c INFO: computed Horn-clause counts
1.90/22.84	c INFO: computed variable graph statistics
1.90/22.84	c INFO: collected features for HOME/instance-3317051-1304267280.cnf in 1.35 s
1.96/22.90	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.38)
1.96/22.93	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.14)
1.96/22.96	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.09)
1.96/22.99	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.07)
401.58/123.10	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.06)
401.96/123.13	c INFO: running gnovelty+2@0 for 97 with 4698 remaining (b = 0.05)
401.96/123.17	c INFO: running FH@0 for 97 with 4698 remaining (b = 0.04)
402.08/123.21	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.03)
801.19/223.27	c INFO: running adaptg2wsat2009++@0 for 97 with 4598 remaining (b = 0.03)
801.56/223.34	c INFO: running FH@0 for 97 with 4598 remaining (b = 0.02)
801.56/223.38	c INFO: running hybridGM3@0 for 194 with 4598 remaining (b = 0.04)
801.71/223.43	c INFO: running gnovelty+2@0 for 97 with 4598 remaining (b = 0.02)
1200.27/323.44	c INFO: running hybridGM3@0 for 194 with 4498 remaining (b = 0.03)
1200.59/323.52	c INFO: running hybridGM3@0 for 194 with 4498 remaining (b = 0.03)
1200.83/323.65	c INFO: running TNM@0 for 1552 with 4497 remaining (b = 0.21)
1583.79/419.76	c INFO: running adaptg2wsat2009++@0 for 97 with 4401 remaining (b = 0.02)
1981.92/519.87	c INFO: running hybridGM3@100 for 97 with 4301 remaining (b = 0.01)
1982.32/519.93	c INFO: running hybridGM3@98 for 97 with 4301 remaining (b = 0.01)
1982.72/520.00	c INFO: running hybridGM3@99 for 97 with 4301 remaining (b = 0.01)
2368.17/617.06	c INFO: running hybridGM3@295 for 97 with 4204 remaining (b = 0.01)
2380.48/620.14	c INFO: running FH@0 for 97 with 4201 remaining (b = 0.01)
2384.82/621.21	c INFO: running adaptg2wsat2009++@0 for 97 with 4200 remaining (b = 0.01)
2770.29/718.26	c INFO: running hybridGM3@396 for 97 with 4103 remaining (b = 0.01)
2771.04/718.41	c INFO: running adaptg2wsat2009++@99 for 97 with 4103 remaining (b = 0.01)
2774.64/719.38	c INFO: running gnovelty+2@0 for 97 with 4102 remaining (b = 0.01)
2779.03/720.48	c INFO: running hybridGM3@0 for 194 with 4101 remaining (b = 0.02)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317051-1304267280/watcher-3317051-1304267280 -o /tmp/evaluation-result-3317051-1304267280/solver-3317051-1304267280 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317051-1304267280.cnf -seed 489172721 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 2/174 27856
/proc/meminfo: memFree=30982536/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=11800 CPUtime=0 cores=0,2,4,6
/proc/27856/stat : 27856 (runsolver) S 27853 27856 27821 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 1 0 519143770 12083200 55 33554432000 4194304 5434984 140736989953200 18446744073709551615 5055495 0 2147483391 4096 24578 18446744071699199166 0 0 17 4 0 0 0
/proc/27856/statm: 2950 55 34 303 0 2644 0

[startup+0.0781941 s]
/proc/loadavg: 0.00 0.00 0.00 2/174 27856
/proc/meminfo: memFree=30982536/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=14384 CPUtime=0 cores=0,2,4,6
/proc/27856/stat : 27856 (python) D 27853 27856 27821 0 -1 4194304 233 0 5 0 0 0 0 0 20 0 1 0 519143770 14729216 169 33554432000 4194304 5683644 140736257898752 18446744073709551615 4689677 0 0 4096 0 18446744071562234166 0 0 17 4 0 0 4
/proc/27856/statm: 3596 169 135 364 0 83 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 14384

[startup+0.101231 s]
/proc/loadavg: 0.00 0.00 0.00 2/174 27856
/proc/meminfo: memFree=30982536/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=14820 CPUtime=0 cores=0,2,4,6
/proc/27856/stat : 27856 (python) R 27853 27856 27821 0 -1 4202496 506 0 6 0 0 0 0 0 20 0 1 0 519143770 15175680 419 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338379621 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 4
/proc/27856/statm: 3705 419 259 364 0 192 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 14820

[startup+0.30117 s]
/proc/loadavg: 0.00 0.00 0.00 2/174 27856
/proc/meminfo: memFree=30982536/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=15188 CPUtime=0 cores=0,2,4,6
/proc/27856/stat : 27856 (python) S 27853 27856 27821 0 -1 4202496 723 0 6 0 0 0 0 0 17 0 1 0 519143770 15552512 618 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338379700 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 6
/proc/27856/statm: 3797 618 322 364 0 284 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15188

[startup+0.701133 s]
/proc/loadavg: 0.00 0.00 0.00 2/174 27856
/proc/meminfo: memFree=30982536/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=15776 CPUtime=0 cores=0,2,4,6
/proc/27856/stat : 27856 (python) S 27853 27856 27821 0 -1 4202496 898 0 6 0 0 0 0 0 15 0 1 0 519143770 16154624 784 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338381824 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 10
/proc/27856/statm: 3944 784 346 364 0 431 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15776

[startup+1.50092 s]
/proc/loadavg: 0.00 0.00 0.00 1/178 27860
/proc/meminfo: memFree=30974764/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=25288 CPUtime=0.05 cores=0,2,4,6
/proc/27856/stat : 27856 (python) S 27853 27856 27821 0 -1 4202496 1335 0 10 0 4 1 0 0 15 0 1 0 519143770 25894912 1191 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338381824 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 16
/proc/27856/statm: 6322 1191 438 364 0 723 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 25288

[startup+3.10055 s]
/proc/loadavg: 0.08 0.02 0.01 1/178 27860
/proc/meminfo: memFree=30971008/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=50016 CPUtime=0.09 cores=0,2,4,6
/proc/27856/stat : 27856 (python) D 27853 27856 27821 0 -1 4202496 1959 0 17 0 7 2 0 0 15 0 1 0 519143770 51216384 1737 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338383616 0 0 16781312 2 18446744071562234166 0 0 17 4 0 0 22
/proc/27856/statm: 12504 1737 480 364 0 1204 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 50016
heavy processes:

[startup+6.30101 s]
/proc/loadavg: 0.08 0.02 0.01 1/178 27860
/proc/meminfo: memFree=30961404/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=99344 CPUtime=0.12 cores=0,2,4,6
/proc/27856/stat : 27856 (python) S 27853 27856 27821 0 -1 4202496 2828 0 29 0 9 3 0 0 16 0 1 0 519143770 101728256 2557 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338379621 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 63
/proc/27856/statm: 24836 2557 818 364 0 1687 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 99344
heavy processes:

[startup+12.7007 s]
/proc/loadavg: 0.15 0.03 0.01 1/178 27868
/proc/meminfo: memFree=30936028/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=148960 CPUtime=0.36 cores=0,2,4,6
/proc/27856/stat : 27856 (python) S 27853 27856 27821 0 -1 4202496 6243 922 46 8 24 12 0 0 15 0 1 0 519143770 152535040 4414 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338381824 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 144
/proc/27856/statm: 37240 4414 1212 364 0 3181 0
Current children cumulated CPU time (s) 0.36
Current children cumulated vsize (KiB) 148960
heavy processes:

[startup+25.501 s]
/proc/loadavg: 0.28 0.06 0.02 9/202 27893
/proc/meminfo: memFree=30837880/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=324040 CPUtime=2.05 cores=0,2,4,6
/proc/27856/stat : 27856 (python) S 27853 27856 27821 0 -1 4202496 15564 922 116 8 184 21 0 0 21 0 5 0 519143770 331816960 8209 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 321
/proc/27856/statm: 81010 8209 1883 364 0 16919 0
[pid=27881] ppid=27856 vsize=283512 CPUtime=0 cores=0,2,4,6
/proc/27881/stat : 27881 (python) S 27856 27856 27821 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 519146061 290316288 6583 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403855 0 0 16781312 514 4358626824519909636 0 0 17 2 0 0 0
/proc/27881/statm: 70878 6583 355 364 0 6796 0
[pid=27883] ppid=27881 vsize=1018996 CPUtime=2.55 cores=0,2,4,6
/proc/27883/stat : 27883 (gnovelty+2) R 27881 27883 27883 0 -1 4202496 962 0 0 0 255 0 0 0 25 0 1 0 519146061 1043451904 698 33554432000 134512640 135663704 4292706752 18446744073709551615 134525157 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/27883/statm: 254749 698 113 282 0 254465 0
[pid=27884] ppid=27856 vsize=293700 CPUtime=0 cores=0,2,4,6
/proc/27884/stat : 27884 (python) S 27856 27856 27821 0 -1 4202560 747 0 0 0 0 0 0 0 18 0 1 0 519146064 300748800 6664 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403910 0 0 16781312 514 1078413653201355012 0 0 17 2 0 0 0
/proc/27884/statm: 73425 6664 349 364 0 9340 0
[pid=27886] ppid=27884 vsize=1018996 CPUtime=2.55 cores=0,2,4,6
/proc/27886/stat : 27886 (gnovelty+2) R 27884 27886 27886 0 -1 4202496 961 0 0 0 255 0 0 0 25 0 1 0 519146064 1043451904 697 33554432000 134512640 135663704 4291580944 18446744073709551615 134524919 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/27886/statm: 254749 697 113 282 0 254465 0
[pid=27887] ppid=27856 vsize=303956 CPUtime=0 cores=0,2,4,6
/proc/27887/stat : 27887 (python) S 27856 27856 27821 0 -1 4202560 741 0 0 0 0 0 0 0 18 0 1 0 519146067 311250944 6668 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403910 0 0 16781312 514 1078402108329263364 0 0 17 4 0 0 0
/proc/27887/statm: 75989 6668 349 364 0 11901 0
[pid=27889] ppid=27887 vsize=421428 CPUtime=2.52 cores=0,2,4,6
/proc/27889/stat : 27889 (adaptg2wsat2009) R 27887 27889 27889 0 -1 4202496 2049 0 0 0 252 0 0 0 25 0 1 0 519146067 431542272 1783 33554432000 134512640 135107411 4290224112 18446744073709551615 134531600 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/27889/statm: 105357 1783 59 146 0 104233 0
[pid=27890] ppid=27856 vsize=314212 CPUtime=0 cores=0,2,4,6
/proc/27890/stat : 27890 (python) S 27856 27856 27821 0 -1 4202560 753 0 0 0 0 0 0 0 18 0 1 0 519146070 321753088 6672 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403910 0 0 16781312 514 4415165636565500163 0 0 17 0 0 0 0
/proc/27890/statm: 78553 6672 349 364 0 14462 0
[pid=27892] ppid=27890 vsize=421428 CPUtime=2.49 cores=0,2,4,6
/proc/27892/stat : 27892 (adaptg2wsat2009) R 27890 27892 27892 0 -1 4202496 2050 0 0 0 249 0 0 0 25 0 1 0 519146070 431542272 1782 33554432000 134512640 135107411 4288495296 18446744073709551615 134532371 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/27892/statm: 105357 1782 59 146 0 104233 0
[pid=27856/tid=27882] ppid=27853 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27882/stat : 27882 (python) S 27853 27856 27821 0 -1 4202560 26 922 0 8 0 0 0 0 19 0 5 0 519146061 331816960 8209 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 4 0 0 0
[pid=27856/tid=27885] ppid=27853 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27885/stat : 27885 (python) S 27853 27856 27821 0 -1 4202560 24 922 0 8 0 0 0 0 21 0 5 0 519146064 331816960 8209 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 0 0 0 0
[pid=27856/tid=27888] ppid=27853 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27888/stat : 27888 (python) S 27853 27856 27821 0 -1 4202560 32 922 0 8 0 0 0 0 20 0 5 0 519146067 331816960 8209 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 0 0 0 0
[pid=27856/tid=27891] ppid=27853 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27891/stat : 27891 (python) S 27853 27856 27821 0 -1 4202560 23 922 0 8 0 0 0 0 22 0 5 0 519146070 331816960 8209 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.16
Current children cumulated vsize (KiB) 4400268
heavy processes:

[startup+51.101 s]
/proc/loadavg: 2.97 0.71 0.24 9/202 27893
/proc/meminfo: memFree=30834164/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=324040 CPUtime=2.05 cores=0,2,4,6
/proc/27856/stat : 27856 (python) S 27853 27856 27821 0 -1 4202496 15564 922 116 8 184 21 0 0 21 0 5 0 519143770 331816960 8209 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 321
/proc/27856/statm: 81010 8209 1883 364 0 16919 0
[pid=27881] ppid=27856 vsize=283512 CPUtime=0.03 cores=0,2,4,6
/proc/27881/stat : 27881 (python) S 27856 27856 27821 0 -1 4202560 905 0 0 0 0 3 0 0 18 0 1 0 519146061 290316288 6606 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403855 0 0 16781312 514 10989240487621132548 0 0 17 2 0 0 0
/proc/27881/statm: 70878 6606 378 364 0 6796 0
[pid=27883] ppid=27881 vsize=1018996 CPUtime=28.01 cores=0,2,4,6
/proc/27883/stat : 27883 (gnovelty+2) R 27881 27883 27883 0 -1 4202496 962 0 0 0 2801 0 0 0 25 0 1 0 519146061 1043451904 698 33554432000 134512640 135663704 4292706752 18446744073709551615 134525070 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/27883/statm: 254749 698 113 282 0 254465 0
[pid=27884] ppid=27856 vsize=293700 CPUtime=0.03 cores=0,2,4,6
/proc/27884/stat : 27884 (python) S 27856 27856 27821 0 -1 4202560 921 0 0 0 2 1 0 0 18 0 1 0 519146064 300748800 6687 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403910 0 0 16781312 514 4584620165320179711 0 0 17 2 0 0 0
/proc/27884/statm: 73425 6687 372 364 0 9340 0
[pid=27886] ppid=27884 vsize=1018996 CPUtime=28.11 cores=0,2,4,6
/proc/27886/stat : 27886 (gnovelty+2) R 27884 27886 27886 0 -1 4202496 961 0 0 0 2811 0 0 0 25 0 1 0 519146064 1043451904 697 33554432000 134512640 135663704 4291580944 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/27886/statm: 254749 697 113 282 0 254465 0
[pid=27887] ppid=27856 vsize=303956 CPUtime=0.03 cores=0,2,4,6
/proc/27887/stat : 27887 (python) S 27856 27856 27821 0 -1 4202560 904 0 0 0 1 2 0 0 18 0 1 0 519146067 311250944 6691 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403910 0 0 16781312 514 10989240487621132548 0 0 17 4 0 0 0
/proc/27887/statm: 75989 6691 372 364 0 11901 0
[pid=27889] ppid=27887 vsize=421428 CPUtime=28.07 cores=0,2,4,6
/proc/27889/stat : 27889 (adaptg2wsat2009) R 27887 27889 27889 0 -1 4202496 2049 0 0 0 2807 0 0 0 25 0 1 0 519146067 431542272 1783 33554432000 134512640 135107411 4290224112 18446744073709551615 134531392 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/27889/statm: 105357 1783 59 146 0 104233 0
[pid=27890] ppid=27856 vsize=314212 CPUtime=0.02 cores=0,2,4,6
/proc/27890/stat : 27890 (python) S 27856 27856 27821 0 -1 4202560 925 0 0 0 1 1 0 0 18 0 1 0 519146070 321753088 6695 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403910 0 0 16781312 514 10989240487621132547 0 0 17 0 0 0 0
/proc/27890/statm: 78553 6695 372 364 0 14462 0
[pid=27892] ppid=27890 vsize=421428 CPUtime=28.01 cores=0,2,4,6
/proc/27892/stat : 27892 (adaptg2wsat2009) R 27890 27892 27892 0 -1 4202496 2050 0 0 0 2801 0 0 0 25 0 1 0 519146070 431542272 1782 33554432000 134512640 135107411 4288495296 18446744073709551615 134531856 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/27892/statm: 105357 1782 59 146 0 104233 0
[pid=27856/tid=27882] ppid=27853 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27882/stat : 27882 (python) S 27853 27856 27821 0 -1 4202560 26 922 0 8 0 0 0 0 19 0 5 0 519146061 331816960 8209 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 4 0 0 0
[pid=27856/tid=27885] ppid=27853 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27885/stat : 27885 (python) S 27853 27856 27821 0 -1 4202560 24 922 0 8 0 0 0 0 21 0 5 0 519146064 331816960 8209 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 0 0 0 0
[pid=27856/tid=27888] ppid=27853 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27888/stat : 27888 (python) S 27853 27856 27821 0 -1 4202560 32 922 0 8 0 0 0 0 20 0 5 0 519146067 331816960 8209 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 0 0 0 0
[pid=27856/tid=27891] ppid=27853 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27891/stat : 27891 (python) S 27853 27856 27821 0 -1 4202560 23 922 0 8 0 0 0 0 22 0 5 0 519146070 331816960 8209 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 114.36
Current children cumulated vsize (KiB) 4400268

[startup+102.307 s]
/proc/loadavg: 6.10 1.97 0.69 9/202 27895
/proc/meminfo: memFree=30834076/32951124 swapFree=67111356/67111528
[pid=27856] ppid=27853 vsize=324040 CPUtime=2.05 cores=0,2,4,6
/proc/27856/stat : 27856 (python) S 27853 27856 27821 0 -1 4202496 15564 922 116 8 184 21 0 0 21 0 5 0 519143770 331816960 8209 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 321
/proc/27856/statm: 81010 8209 1883 364 0 16919 0
[pid=27881] ppid=27856 vsize=283512 CPUtime=0.1 cores=0,2,4,6
/proc/27881/stat : 27881 (python) S 27856 27856 27821 0 -1 4202560 920 0 0 0 4 6 0 0 18 0 1 0 519146061 290316288 6606 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403855 0 0 16781312 514 10989240487621132548 0 0 17 2 0 0 0
/proc/27881/statm: 70878 6606 378 364 0 6796 0
[pid=27883] ppid=27881 vsize=1018996 CPUtime=78.93 cores=0,2,4,6
/proc/27883/stat : 27883 (gnovelty+2) R 27881 27883 27883 0 -1 4202496 962 0 0 0 7893 0 0 0 25 0 1 0 519146061 1043451904 698 33554432000 134512640 135663704 4292706752 18446744073709551615 134525278 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/27883/statm: 254749 698 113 282 0 254465 0
[pid=27884] ppid=27856 vsize=293700 CPUtime=0.1 cores=0,2,4,6
/proc/27884/stat : 27884 (python) S 27856 27856 27821 0 -1 4202560 939 0 0 0 6 4 0 0 18 0 1 0 519146064 300748800 6687 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403910 0 0 16781312 514 10989240487621132548 0 0 17 2 0 0 0

################
# More data... #
################

[pid=27892] ppid=27890 vsize=421428 CPUtime=101.84 cores=0,2,4,6
/proc/27892/stat : 27892 (adaptg2wsat2009) R 27890 27892 27892 0 -1 4202496 2050 0 0 0 10184 0 0 0 21 0 1 0 519146070 431542272 1782 33554432000 134512640 135107411 4288495296 18446744073709551615 134532349 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/27892/statm: 105357 1782 59 146 0 104233 0
[pid=27890/tid=27917] ppid=27856 vsize=314212 CPUtime=0 cores=0,2,4,6
/proc/27890/task/27917/stat : 27917 (python) S 27856 27856 27821 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 519156082 321753088 6724 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453940679104 0 0 -1 4 0 0 0
[pid=27914] ppid=27856 vsize=324468 CPUtime=0.13 cores=0,2,4,6
/proc/27914/stat : 27914 (python) S 27856 27856 27821 0 -1 4202560 1123 0 0 0 7 6 0 0 19 0 2 0 519156081 332255232 6728 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/27914/statm: 81117 6728 400 364 0 17023 0
[pid=27918] ppid=27914 vsize=421428 CPUtime=99.85 cores=0,2,4,6
/proc/27918/stat : 27918 (adaptg2wsat2009) T 27914 27918 27918 0 -1 4202496 2049 0 0 0 9984 1 0 0 25 0 1 0 519156082 431542272 1782 33554432000 134512640 135107411 4294899472 18446744073709551615 134537745 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/27918/statm: 105357 1782 59 146 0 104233 0
[pid=27914/tid=28020] ppid=27856 vsize=324468 CPUtime=0 cores=0,2,4,6
/proc/27914/task/28020/stat : 28020 (python) S 27856 27856 27821 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 519166094 332255232 6728 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453934954496 0 0 -1 0 0 0 0
[pid=27919] ppid=27856 vsize=334808 CPUtime=0.13 cores=0,2,4,6
/proc/27919/stat : 27919 (python) S 27856 27856 27821 0 -1 4202560 1141 0 0 0 6 7 0 0 19 0 2 0 519156084 342843392 6732 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/27919/statm: 83702 6732 400 364 0 19605 0
[pid=27921] ppid=27919 vsize=1018996 CPUtime=100.15 cores=0,2,4,6
/proc/27921/stat : 27921 (gnovelty+2) T 27919 27921 27921 0 -1 4202496 962 0 0 0 10015 0 0 0 25 0 1 0 519156084 1043451904 698 33554432000 134512640 135663704 4286977696 18446744073709551615 134524575 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/27921/statm: 254749 698 113 282 0 254465 0
[pid=27919/tid=28024] ppid=27856 vsize=334808 CPUtime=0 cores=0,2,4,6
/proc/27919/task/28024/stat : 28024 (python) S 27856 27856 27821 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 519166100 342843392 6732 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453940681792 0 0 -1 6 0 0 0
[pid=27922] ppid=27856 vsize=345244 CPUtime=0.12 cores=0,2,4,6
/proc/27922/stat : 27922 (python) S 27856 27856 27821 0 -1 4202560 1190 0 0 0 7 5 0 0 19 0 2 0 519156088 353529856 6746 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/27922/statm: 86311 6746 406 364 0 22211 0
[pid=27924] ppid=27922 vsize=251292 CPUtime=98.88 cores=0,2,4,6
/proc/27924/stat : 27924 (FH) T 27922 27924 27924 0 -1 4202496 1951 0 0 0 9886 2 0 0 25 0 1 0 519156088 257323008 1683 33554432000 134512640 134936988 4291182976 18446744073709551615 134545221 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/27924/statm: 62823 1683 41 104 0 62327 0
[pid=27922/tid=28025] ppid=27856 vsize=345244 CPUtime=0 cores=0,2,4,6
/proc/27922/task/28025/stat : 28025 (python) S 27856 27856 27821 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 519166103 353529856 6746 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453948811456 0 0 -1 6 0 0 0
[pid=27925] ppid=27856 vsize=355320 CPUtime=0.13 cores=0,2,4,6
/proc/27925/stat : 27925 (python) S 27856 27856 27821 0 -1 4202560 1167 0 0 0 7 6 0 0 19 0 2 0 519156092 363847680 6795 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/27925/statm: 88830 6795 406 364 0 24727 0
[pid=27927] ppid=27925 vsize=421428 CPUtime=100.1 cores=0,2,4,6
/proc/27927/stat : 27927 (adaptg2wsat2009) T 27925 27927 27927 0 -1 4202496 2051 0 0 0 10008 2 0 0 25 0 1 0 519156092 431542272 1782 33554432000 134512640 135107411 4290590000 18446744073709551615 134531724 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/27927/statm: 105357 1782 59 146 0 104233 0
[pid=27925/tid=28026] ppid=27856 vsize=355320 CPUtime=0 cores=0,2,4,6
/proc/27925/task/28026/stat : 28026 (python) S 27856 27856 27821 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519166104 363847680 6795 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453948812352 0 0 -1 0 0 0 0
[pid=28021] ppid=27856 vsize=365784 CPUtime=0.13 cores=0,2,4,6
/proc/28021/stat : 28021 (python) S 27856 27856 27821 0 -1 4202560 1155 0 0 0 8 5 0 0 19 0 2 0 519166097 374562816 6793 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/28021/statm: 91446 6793 400 364 0 27340 0
[pid=28023] ppid=28021 vsize=421428 CPUtime=100.1 cores=0,2,4,6
/proc/28023/stat : 28023 (adaptg2wsat2009) T 28021 28023 28023 0 -1 4202496 2054 0 0 0 10010 0 0 0 25 0 1 0 519166098 431542272 1782 33554432000 134512640 135107411 4291917616 18446744073709551615 134531537 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/28023/statm: 105357 1782 59 146 0 104233 0
[pid=28021/tid=28054] ppid=27856 vsize=365784 CPUtime=0 cores=0,2,4,6
/proc/28021/task/28054/stat : 28054 (python) S 27856 27856 27821 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519176110 374562816 6793 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453934952704 0 0 -1 0 0 0 0
[pid=28027] ppid=27856 vsize=376152 CPUtime=0.13 cores=0,2,4,6
/proc/28027/stat : 28027 (python) S 27856 27856 27821 0 -1 4202560 1169 0 0 0 7 6 0 0 19 0 2 0 519166105 385179648 6804 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/28027/statm: 94038 6804 400 364 0 29929 0
[pid=28029] ppid=28027 vsize=251292 CPUtime=98.29 cores=0,2,4,6
/proc/28029/stat : 28029 (FH) T 28027 28029 28029 0 -1 4202496 1953 0 0 0 9829 0 0 0 25 0 1 0 519166105 257323008 1683 33554432000 134512640 134936988 4288105712 18446744073709551615 134554594 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/28029/statm: 62823 1683 41 104 0 62327 0
[pid=28027/tid=28058] ppid=27856 vsize=376152 CPUtime=0 cores=0,2,4,6
/proc/28027/task/28058/stat : 28058 (python) S 27856 27856 27821 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519176118 385179648 6804 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453962085632 0 0 -1 6 0 0 0
[pid=28030] ppid=27856 vsize=386208 CPUtime=0.62 cores=0,2,4,6
/proc/28030/stat : 28030 (python) S 27856 27856 27821 0 -1 4202560 1196 0 0 0 34 28 0 0 18 0 2 0 519166109 395476992 6836 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403910 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/28030/statm: 96552 6836 400 364 0 32440 0
[pid=28032] ppid=28030 vsize=268424 CPUtime=393.8 cores=0,2,4,6
/proc/28032/stat : 28032 (hybridGM3) R 28030 28032 28032 0 -1 4202496 5813 0 0 0 39379 1 0 0 25 0 1 0 519166109 274866176 5543 33554432000 134512640 135204385 4289386272 18446744073709551615 134538520 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/28032/statm: 67106 5543 63 169 0 66936 0
[pid=28030/tid=28085] ppid=27856 vsize=386208 CPUtime=0 cores=0,2,4,6
/proc/28030/task/28085/stat : 28085 (python) S 27856 27856 27821 0 -1 4202560 45 0 0 0 0 0 0 0 18 0 2 0 519185741 395476992 6836 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453934953600 0 0 -1 0 0 0 0
[pid=28033] ppid=27856 vsize=396716 CPUtime=0.14 cores=0,2,4,6
/proc/28033/stat : 28033 (python) S 27856 27856 27821 0 -1 4202560 1171 0 0 0 8 6 0 0 19 0 2 0 519166114 406237184 6840 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/28033/statm: 99179 6840 400 364 0 35064 0
[pid=28035] ppid=28033 vsize=1018996 CPUtime=99.88 cores=0,2,4,6
/proc/28035/stat : 28035 (gnovelty+2) T 28033 28035 28035 0 -1 4202496 968 0 0 0 9987 1 0 0 25 0 1 0 519166114 1043451904 697 33554432000 134512640 135663704 4288490336 18446744073709551615 134544225 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/28035/statm: 254749 697 113 282 0 254465 0
[pid=28033/tid=28062] ppid=27856 vsize=396716 CPUtime=0 cores=0,2,4,6
/proc/28033/task/28062/stat : 28062 (python) S 27856 27856 27821 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 519176131 406237184 6840 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453962087424 0 0 -1 6 0 0 0
[pid=28055] ppid=27856 vsize=407108 CPUtime=0.48 cores=0,2,4,6
/proc/28055/stat : 28055 (python) S 27856 27856 27821 0 -1 4202560 1192 0 0 0 26 22 0 0 18 0 2 0 519176115 416878592 6857 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/28055/statm: 101777 6857 400 364 0 37659 0
[pid=28057] ppid=28055 vsize=268420 CPUtime=296.51 cores=0,2,4,6
/proc/28057/stat : 28057 (hybridGM3) T 28055 28057 28057 0 -1 4202496 5813 0 0 0 29651 0 0 0 25 0 1 0 519176115 274862080 5543 33554432000 134512640 135204385 4293229616 18446744073709551615 134538427 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/28057/statm: 67105 5543 63 169 0 66935 0
[pid=28055/tid=28104] ppid=27856 vsize=407108 CPUtime=0 cores=0,2,4,6
/proc/28055/task/28104/stat : 28104 (python) S 27856 27856 27821 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 519195752 416878592 6857 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453927434944 0 0 -1 4 0 0 0
[pid=28059] ppid=27856 vsize=417276 CPUtime=0.48 cores=0,2,4,6
/proc/28059/stat : 28059 (python) S 27856 27856 27821 0 -1 4202560 1218 0 0 0 25 23 0 0 18 0 2 0 519176123 427290624 6895 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/28059/statm: 104319 6895 400 364 0 40198 0
[pid=28061] ppid=28059 vsize=268424 CPUtime=296.7 cores=0,2,4,6
/proc/28061/stat : 28061 (hybridGM3) T 28059 28061 28061 0 -1 4202496 5818 0 0 0 29668 2 0 0 25 0 1 0 519176123 274866176 5543 33554432000 134512640 135204385 4290399440 18446744073709551615 134538284 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/28061/statm: 67106 5543 63 169 0 66936 0
[pid=28059/tid=28105] ppid=27856 vsize=417276 CPUtime=0 cores=0,2,4,6
/proc/28059/task/28105/stat : 28105 (python) S 27856 27856 27821 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 519195759 427290624 6895 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453927432256 0 0 -1 0 0 0 0
[pid=28064] ppid=27856 vsize=427680 CPUtime=0.64 cores=0,2,4,6
/proc/28064/stat : 28064 (python) S 27856 27856 27821 0 -1 4202560 1077 0 0 0 32 32 0 0 18 0 1 0 519176136 437944320 6877 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403910 0 0 16781312 514 10989240487621132545 0 0 17 6 0 0 0
/proc/28064/statm: 106920 6877 379 364 0 42796 0
[pid=28066] ppid=28064 vsize=459768 CPUtime=391.92 cores=0,2,4,6
/proc/28066/stat : 28066 (TNM) R 28064 28066 28066 0 -1 4202496 1928 0 0 0 39192 0 0 0 25 0 1 0 519176136 470802432 1653 33554432000 134512640 135008748 4287673184 18446744073709551615 134527503 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/28066/statm: 114942 1653 46 122 0 113842 0
[pid=28086] ppid=27856 vsize=437832 CPUtime=0.17 cores=0,2,4,6
/proc/28086/stat : 28086 (python) S 27856 27856 27821 0 -1 4202560 1182 0 0 0 9 8 0 0 19 0 2 0 519185746 448339968 6925 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/28086/statm: 109458 6925 407 364 0 45331 0
[pid=28088] ppid=28086 vsize=421428 CPUtime=97.4 cores=0,2,4,6
/proc/28088/stat : 28088 (adaptg2wsat2009) T 28086 28088 28088 0 -1 4202496 2060 0 0 0 9738 2 0 0 25 0 1 0 519185747 431542272 1783 33554432000 134512640 135107411 4291441360 18446744073709551615 134531435 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/28088/statm: 105357 1783 59 146 0 104233 0
[pid=28086/tid=28106] ppid=27856 vsize=437832 CPUtime=0 cores=0,2,4,6
/proc/28086/task/28106/stat : 28106 (python) S 27856 27856 27821 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 519195765 448339968 6925 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453904988992 0 0 -1 6 0 0 0
[pid=28114] ppid=27856 vsize=448372 CPUtime=0.16 cores=0,2,4,6
/proc/28114/stat : 28114 (python) S 27856 27856 27821 0 -1 4202560 1224 0 0 0 8 8 0 0 19 0 2 0 519205785 459132928 6938 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350210507 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/28114/statm: 112093 6938 401 364 0 47963 0
[pid=28116] ppid=28114 vsize=251288 CPUtime=98.96 cores=0,2,4,6
/proc/28116/stat : 28116 (FH) T 28114 28116 28116 0 -1 4202496 1958 0 0 0 9895 1 0 0 25 0 1 0 519205785 257318912 1682 33554432000 134512640 134936988 4293370928 18446744073709551615 134545336 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/28116/statm: 62822 1682 41 104 0 62326 0
[pid=28114/tid=28174] ppid=27856 vsize=448372 CPUtime=0 cores=0,2,4,6
/proc/28114/task/28174/stat : 28174 (python) S 27856 27856 27821 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519215704 459132928 6938 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 514 18446604453919791936 0 0 -1 0 0 0 0
[pid=28175] ppid=27856 vsize=458652 CPUtime=0 cores=0,2,4,6
/proc/28175/stat : 28175 (python) S 27856 27856 27821 0 -1 4202560 846 0 0 0 0 0 0 0 19 0 1 0 519215709 469659648 6899 33554432000 4194304 5683644 140736257898752 18446744073709551615 270338403910 0 0 16781312 514 4341233100324339711 0 0 17 6 0 0 0
/proc/28175/statm: 114663 6899 350 364 0 50530 0
[pid=28177] ppid=28175 vsize=1018996 CPUtime=0.99 cores=0,2,4,6
/proc/28177/stat : 28177 (gnovelty+2) R 28175 28177 28177 0 -1 4202496 972 0 0 0 99 0 0 0 23 0 1 0 519215709 1043451904 697 33554432000 134512640 135663704 4293863888 18446744073709551615 134525691 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/28177/statm: 254749 697 113 282 0 254465 0
[pid=27856/tid=27882] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27882/stat : 27882 (python) S 27853 27856 27821 0 -1 4202560 26 922 0 8 0 0 0 0 19 0 19 0 519146061 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 4 0 0 0
[pid=27856/tid=27885] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27885/stat : 27885 (python) S 27853 27856 27821 0 -1 4202560 24 922 0 8 0 0 0 0 21 0 19 0 519146064 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 0 0 0 0
[pid=27856/tid=27888] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27888/stat : 27888 (python) S 27853 27856 27821 0 -1 4202560 36 922 0 8 0 0 0 0 18 0 19 0 519146067 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 0 0 0 0
[pid=27856/tid=27891] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27891/stat : 27891 (python) S 27853 27856 27821 0 -1 4202560 27 922 0 8 0 0 0 0 18 0 19 0 519146070 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 0 0 0 0
[pid=27856/tid=27915] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27915/stat : 27915 (python) S 27853 27856 27821 0 -1 4202560 25 922 0 8 0 0 0 0 20 0 19 0 519156081 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 2 0 0 0
[pid=27856/tid=27920] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27920/stat : 27920 (python) S 27853 27856 27821 0 -1 4202560 26 922 0 8 0 0 0 0 20 0 19 0 519156084 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 4 0 0 0
[pid=27856/tid=27923] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27923/stat : 27923 (python) S 27853 27856 27821 0 -1 4202560 21 922 0 8 0 0 0 0 20 0 19 0 519156088 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 6 0 0 0
[pid=27856/tid=27926] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/27926/stat : 27926 (python) S 27853 27856 27821 0 -1 4202560 28 922 0 8 0 0 0 0 20 0 19 0 519156092 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 0 0 0 0
[pid=27856/tid=28022] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/28022/stat : 28022 (python) S 27853 27856 27821 0 -1 4202560 28 922 0 8 0 0 0 0 20 0 19 0 519166098 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 0 0 0 0
[pid=27856/tid=28028] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/28028/stat : 28028 (python) S 27853 27856 27821 0 -1 4202560 25 922 0 8 0 0 0 0 21 0 19 0 519166105 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 2 0 0 0
[pid=27856/tid=28031] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/28031/stat : 28031 (python) S 27853 27856 27821 0 -1 4202560 44 922 0 8 0 0 0 0 18 0 19 0 519166109 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 6 0 0 0
[pid=27856/tid=28034] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/28034/stat : 28034 (python) S 27853 27856 27821 0 -1 4202560 26 922 0 8 0 0 0 0 22 0 19 0 519166114 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 6 0 0 0
[pid=27856/tid=28056] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/28056/stat : 28056 (python) S 27853 27856 27821 0 -1 4202560 32 922 0 8 0 0 0 0 18 0 19 0 519176115 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 6 0 0 0
[pid=27856/tid=28060] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/28060/stat : 28060 (python) S 27853 27856 27821 0 -1 4202560 30 922 0 8 0 0 0 0 18 0 19 0 519176123 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 6 0 0 0
[pid=27856/tid=28065] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/28065/stat : 28065 (python) S 27853 27856 27821 0 -1 4202560 26 922 0 8 0 0 0 0 22 0 19 0 519176136 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 6 0 0 0
[pid=27856/tid=28087] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/28087/stat : 28087 (python) S 27853 27856 27821 0 -1 4202560 26 922 0 8 0 0 0 0 20 0 19 0 519185747 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 6 0 0 0
[pid=27856/tid=28115] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/28115/stat : 28115 (python) S 27853 27856 27821 0 -1 4202560 26 922 0 8 0 0 0 0 20 0 19 0 519205785 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 6 0 0 0
[pid=27856/tid=28176] ppid=27853 vsize=468304 CPUtime=0 cores=0,2,4,6
/proc/27856/task/28176/stat : 28176 (python) S 27853 27856 27821 0 -1 4202560 26 922 0 8 0 0 0 0 20 0 19 0 519215709 479543296 8464 33554432000 4194304 5683644 140736257898752 18446744073709551615 270350207409 0 0 16781312 2 18446604440279302272 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2779.03
Current children cumulated vsize (KiB) 16798004

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 27856 and gives
#  childrusage.ru_utime.tv_sec=499
#  childrusage.ru_utime.tv_usec=755025
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=403786
# CPU time returned by wait4() is 501.159
# while last known CPU time is 2779.03
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 720.518
CPU time (s): 2779.03
CPU user time (s): 2776.72
CPU system time (s): 2.31
CPU usage (%): 385.699
Max. virtual memory (cumulated for all children) (KiB): 16798004

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 499.755
system time used= 1.40379
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 53828
page faults= 124
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 21518
involuntary context switches= 2617

runsolver used 4.30635 second user time and 8.07777 second system time

The end

Launcher Data 

Begin job on node143 at 2011-05-01 18:28:00
IDJOB=3317051
IDBENCH=82516
IDSOLVER=1705
FILE ID=node143/3317051-1304267280
RUNJOBID= node143-1304267279-27839
PBS_JOBID= 13173936
Free space on /tmp= 73584 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k5-r20-v1500-c30000-S976428817-077-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317051-1304267280/watcher-3317051-1304267280 -o /tmp/evaluation-result-3317051-1304267280/solver-3317051-1304267280 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317051-1304267280.cnf -seed 489172721 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4d2cfd51677809bb15d90c916352e4e2
RANDOM SEED=489172721

node143.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30983088 kB
Buffers:        294440 kB
Cached:        1321400 kB
SwapCached:        168 kB
Active:         484812 kB
Inactive:      1195184 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30983088 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:           64848 kB
Writeback:           0 kB
AnonPages:       63760 kB
Mapped:          14628 kB
Slab:           224568 kB
PageTables:       4628 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   180708 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73512 MiB
End job on node143 at 2011-05-01 18:40:04