Trace number 3317077

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 2364.02 611.719

General information on the benchmark

Namerandom/large/
unif-k7-r85-v300-c25500-S942666749-061-UNKNOWN.cnf
MD5SUM456ae67a44e812130272597099547869
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables300
Number of clauses25500
Sum of the clauses size178500
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 525500

Solver Data 

0.54/16.87	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.54/16.87	c INFO: solving HOME/instance-3317077-1304267285.cnf
0.54/16.87	c INFO: computing task features
0.67/17.04	c INFO: computed variable-clause graph statistics
0.67/17.05	c INFO: computed clause balance statistics
1.58/17.97	c INFO: computed variable balance statistics
1.98/18.34	c INFO: computed small-clause counts
1.98/18.34	c INFO: computed Horn-clause counts
1.98/18.37	c INFO: computed variable graph statistics
1.98/18.37	c INFO: collected features for HOME/instance-3317077-1304267285.cnf in 1.50 s
2.07/18.41	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.37)
2.07/18.44	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.13)
2.07/18.47	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.08)
2.16/18.50	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.07)
398.11/117.64	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.05)
398.11/117.68	c INFO: running gnovelty+2@0 for 97 with 4699 remaining (b = 0.04)
402.10/118.60	c INFO: running FH@0 for 97 with 4698 remaining (b = 0.03)
402.10/118.64	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.03)
797.35/217.81	c INFO: running adaptg2wsat2009++@0 for 97 with 4599 remaining (b = 0.02)
797.35/217.86	c INFO: running FH@0 for 97 with 4599 remaining (b = 0.02)
797.35/217.90	c INFO: running hybridGM3@0 for 194 with 4598 remaining (b = 0.04)
798.38/218.77	c INFO: running gnovelty+2@0 for 97 with 4598 remaining (b = 0.02)
1196.04/318.00	c INFO: running hybridGM3@0 for 194 with 4498 remaining (b = 0.03)
1196.42/318.05	c INFO: running adaptg2wsat2009++@0 for 97 with 4498 remaining (b = 0.01)
1198.26/319.00	c INFO: running hybridGM3@0 for 194 with 4497 remaining (b = 0.03)
1595.50/418.24	c INFO: running hybridGM3@0 for 194 with 4398 remaining (b = 0.02)
1599.85/419.31	c INFO: running hybridGM3@99 for 97 with 4397 remaining (b = 0.01)
1977.77/514.40	c INFO: running hybridGM3@100 for 97 with 4302 remaining (b = 0.01)
1978.17/514.50	c INFO: running hybridGM3@294 for 97 with 4302 remaining (b = 0.01)
1981.75/515.40	c INFO: running hybridGM3@99 for 97 with 4301 remaining (b = 0.01)
2356.06/609.69	c INFO: running TNM@0 for 1552 with 4207 remaining (b = 0.19)
2359.64/610.59	c INFO: running FH@0 for 97 with 4206 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317077-1304267285/watcher-3317077-1304267285 -o /tmp/evaluation-result-3317077-1304267285/solver-3317077-1304267285 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317077-1304267285.cnf -seed 40564498 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 4/176 11023
/proc/meminfo: memFree=23305052/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=11800 CPUtime=0 cores=1,3,5,7
/proc/11023/stat : 11023 (runsolver) R 11019 11023 10986 0 -1 4202560 46 0 0 0 0 0 0 0 20 0 1 0 616959545 12083200 55 33554432000 4194304 5434984 140733747872032 18446744073709551615 5055495 0 0 4096 24578 0 0 0 17 3 0 0 0
/proc/11023/statm: 2950 55 34 303 0 2644 0

[startup+0.103082 s]
/proc/loadavg: 0.00 0.00 0.00 4/176 11023
/proc/meminfo: memFree=23305052/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=15080 CPUtime=0 cores=1,3,5,7
/proc/11023/stat : 11023 (python) S 11019 11023 10986 0 -1 4202496 610 0 6 0 0 0 0 0 17 0 1 0 616959545 15441920 512 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735176704 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 3
/proc/11023/statm: 3770 512 300 364 0 257 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15080

[startup+0.201091 s]
/proc/loadavg: 0.00 0.00 0.00 4/176 11023
/proc/meminfo: memFree=23305052/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=15184 CPUtime=0 cores=1,3,5,7
/proc/11023/stat : 11023 (python) S 11019 11023 10986 0 -1 4202496 706 0 6 0 0 0 0 0 16 0 1 0 616959545 15548416 603 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735176704 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 5
/proc/11023/statm: 3796 603 319 364 0 283 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15184

[startup+0.301047 s]
/proc/loadavg: 0.00 0.00 0.00 4/176 11023
/proc/meminfo: memFree=23305052/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=15188 CPUtime=0 cores=1,3,5,7
/proc/11023/stat : 11023 (python) S 11019 11023 10986 0 -1 4202496 719 0 6 0 0 0 0 0 16 0 1 0 616959545 15552512 615 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735174580 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 6
/proc/11023/statm: 3797 615 321 364 0 284 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15188

[startup+0.700928 s]
/proc/loadavg: 0.00 0.00 0.00 4/176 11023
/proc/meminfo: memFree=23305052/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=15380 CPUtime=0 cores=1,3,5,7
/proc/11023/stat : 11023 (python) D 11019 11023 10986 0 -1 4202496 789 0 6 0 0 0 0 0 16 0 1 0 616959545 15749120 681 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735176704 0 0 16781312 2 0 0 0 17 7 0 0 32
/proc/11023/statm: 3845 681 322 364 0 332 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15380

[startup+1.50084 s]
/proc/loadavg: 0.00 0.00 0.00 2/178 11025
/proc/meminfo: memFree=23298940/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=15904 CPUtime=0 cores=1,3,5,7
/proc/11023/stat : 11023 (python) S 11019 11023 10986 0 -1 4202496 952 0 6 0 0 0 0 0 16 0 1 0 616959545 16285696 833 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735174501 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 48
/proc/11023/statm: 3976 833 357 364 0 463 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15904

[startup+3.10054 s]
/proc/loadavg: 0.16 0.03 0.01 2/178 11025
/proc/meminfo: memFree=23296776/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=43616 CPUtime=0.07 cores=1,3,5,7
/proc/11023/stat : 11023 (python) S 11019 11023 10986 0 -1 4202496 1863 0 16 0 5 2 0 0 15 0 1 0 616959545 44662784 1653 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735176704 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 110
/proc/11023/statm: 10904 1653 467 364 0 1169 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 43616
heavy processes:

[startup+6.30099 s]
/proc/loadavg: 0.16 0.03 0.01 2/178 11025
/proc/meminfo: memFree=23278348/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=107296 CPUtime=0.15 cores=1,3,5,7
/proc/11023/stat : 11023 (python) S 11019 11023 10986 0 -1 4202496 3328 0 32 0 10 5 0 0 16 0 1 0 616959545 109871104 3019 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735176704 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 130
/proc/11023/statm: 26824 3019 871 364 0 2055 0
Current children cumulated CPU time (s) 0.15
Current children cumulated vsize (KiB) 107296
heavy processes:

[startup+12.7009 s]
/proc/loadavg: 0.23 0.05 0.02 2/178 11033
/proc/meminfo: memFree=23262616/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=135900 CPUtime=0.33 cores=1,3,5,7
/proc/11023/stat : 11023 (python) D 11019 11023 10986 0 -1 4202496 6011 927 44 8 23 10 0 0 15 0 1 0 616959545 139161600 4192 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735178496 0 0 16781312 2 18446744071562234166 0 0 17 7 0 0 199
/proc/11023/statm: 33975 4192 1032 364 0 3161 0
Current children cumulated CPU time (s) 0.33
Current children cumulated vsize (KiB) 135900
heavy processes:

[startup+25.5006 s]
/proc/loadavg: 0.96 0.21 0.07 10/202 11058
/proc/meminfo: memFree=23164680/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=321288 CPUtime=2.16 cores=1,3,5,7
/proc/11023/stat : 11023 (python) S 11019 11023 10986 0 -1 4202496 15277 927 114 8 198 18 0 0 24 0 5 0 616959545 328998912 7538 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 310
/proc/11023/statm: 80322 7538 1883 364 0 16231 0
[pid=11047] ppid=11023 vsize=280760 CPUtime=0 cores=1,3,5,7
/proc/11047/stat : 11047 (python) S 11023 11023 10986 0 -1 4202560 887 0 0 0 0 0 0 0 18 0 1 0 616961386 287498240 6001 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198735 0 0 16781312 514 5629746103990583560 0 0 17 3 0 0 0
/proc/11047/statm: 70190 6001 378 364 0 6108 0
[pid=11049] ppid=11047 vsize=1019708 CPUtime=7.07 cores=1,3,5,7
/proc/11049/stat : 11049 (gnovelty+2) R 11047 11049 11049 0 -1 4202496 955 0 0 0 707 0 0 0 25 0 1 0 616961386 1044180992 690 33554432000 134512640 135663704 4289265600 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/11049/statm: 254927 690 113 282 0 254643 0
[pid=11050] ppid=11023 vsize=291084 CPUtime=0 cores=1,3,5,7
/proc/11050/stat : 11050 (python) S 11023 11023 10986 0 -1 4202560 896 0 0 0 0 0 0 0 18 0 1 0 616961389 298070016 5999 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198790 0 0 16781312 514 5629749952281280776 0 0 17 7 0 0 0
/proc/11050/statm: 72771 5999 372 364 0 8686 0
[pid=11052] ppid=11050 vsize=1019708 CPUtime=7.04 cores=1,3,5,7
/proc/11052/stat : 11052 (gnovelty+2) R 11050 11052 11052 0 -1 4202496 953 0 0 0 704 0 0 0 25 0 1 0 616961389 1044180992 689 33554432000 134512640 135663704 4290522608 18446744073709551615 134524868 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/11052/statm: 254927 689 113 282 0 254643 0
[pid=11053] ppid=11023 vsize=301272 CPUtime=0 cores=1,3,5,7
/proc/11053/stat : 11053 (python) S 11023 11023 10986 0 -1 4202560 894 0 0 0 0 0 0 0 18 0 1 0 616961392 308502528 6020 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198790 0 0 16781312 514 10652168434869043464 0 0 17 1 0 0 0
/proc/11053/statm: 75318 6020 372 364 0 11230 0
[pid=11055] ppid=11053 vsize=420220 CPUtime=7.01 cores=1,3,5,7
/proc/11055/stat : 11055 (adaptg2wsat2009) R 11053 11055 11055 0 -1 4202496 1668 0 0 0 701 0 0 0 25 0 1 0 616961392 430305280 1400 33554432000 134512640 135107411 4288500720 18446744073709551615 134532107 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/11055/statm: 105055 1400 59 146 0 103931 0
[pid=11056] ppid=11023 vsize=312136 CPUtime=0 cores=1,3,5,7
/proc/11056/stat : 11056 (python) S 11023 11023 10986 0 -1 4202560 931 0 0 0 0 0 0 0 18 0 1 0 616961395 319627264 6024 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198790 0 0 16781312 514 4450525105881055496 0 0 17 7 0 0 0
/proc/11056/statm: 78034 6024 372 364 0 13943 0
[pid=11058] ppid=11056 vsize=420220 CPUtime=6.93 cores=1,3,5,7
/proc/11058/stat : 11058 (adaptg2wsat2009) R 11056 11058 11058 0 -1 4202496 1662 0 0 0 693 0 0 0 25 0 1 0 616961395 430305280 1399 33554432000 134512640 135107411 4288970480 18446744073709551615 134531935 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/11058/statm: 105055 1399 59 146 0 103931 0
[pid=11023/tid=11048] ppid=11019 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11048/stat : 11048 (python) S 11019 11023 10986 0 -1 4202560 23 927 0 8 0 0 0 0 19 0 5 0 616961386 328998912 7538 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 1 0 0 0
[pid=11023/tid=11051] ppid=11019 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11051/stat : 11051 (python) S 11019 11023 10986 0 -1 4202560 23 927 0 8 0 0 0 0 20 0 5 0 616961389 328998912 7538 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11054] ppid=11019 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11054/stat : 11054 (python) S 11019 11023 10986 0 -1 4202560 22 927 0 8 0 0 0 0 22 0 5 0 616961392 328998912 7538 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11057] ppid=11019 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11057/stat : 11057 (python) S 11019 11023 10986 0 -1 4202560 25 927 0 8 0 0 0 0 24 0 5 0 616961395 328998912 7538 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 30.21
Current children cumulated vsize (KiB) 4386396

[startup+51.1011 s]
/proc/loadavg: 3.36 0.84 0.28 10/202 11058
/proc/meminfo: memFree=23163408/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=321288 CPUtime=2.16 cores=1,3,5,7
/proc/11023/stat : 11023 (python) S 11019 11023 10986 0 -1 4202496 15277 927 114 8 198 18 0 0 24 0 5 0 616959545 328998912 7538 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 310
/proc/11023/statm: 80322 7538 1883 364 0 16231 0
[pid=11047] ppid=11023 vsize=280760 CPUtime=0.04 cores=1,3,5,7
/proc/11047/stat : 11047 (python) S 11023 11023 10986 0 -1 4202560 897 0 0 0 3 1 0 0 18 0 1 0 616961386 287498240 6001 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198735 0 0 16781312 514 16926972713518465288 0 0 17 3 0 0 0
/proc/11047/statm: 70190 6001 378 364 0 6108 0
[pid=11049] ppid=11047 vsize=1019708 CPUtime=32.67 cores=1,3,5,7
/proc/11049/stat : 11049 (gnovelty+2) R 11047 11049 11049 0 -1 4202496 955 0 0 0 3267 0 0 0 25 0 1 0 616961386 1044180992 690 33554432000 134512640 135663704 4289265600 18446744073709551615 134545190 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/11049/statm: 254927 690 113 282 0 254643 0
[pid=11050] ppid=11023 vsize=291084 CPUtime=0.03 cores=1,3,5,7
/proc/11050/stat : 11050 (python) S 11023 11023 10986 0 -1 4202560 905 0 0 0 2 1 0 0 18 0 1 0 616961389 298070016 5999 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198790 0 0 16781312 514 16926972713518465288 0 0 17 7 0 0 0
/proc/11050/statm: 72771 5999 372 364 0 8686 0
[pid=11052] ppid=11050 vsize=1019708 CPUtime=32.61 cores=1,3,5,7
/proc/11052/stat : 11052 (gnovelty+2) R 11050 11052 11052 0 -1 4202496 953 0 0 0 3261 0 0 0 25 0 1 0 616961389 1044180992 689 33554432000 134512640 135663704 4290522608 18446744073709551615 134524455 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/11052/statm: 254927 689 113 282 0 254643 0
[pid=11053] ppid=11023 vsize=301272 CPUtime=0.03 cores=1,3,5,7
/proc/11053/stat : 11053 (python) S 11023 11023 10986 0 -1 4202560 904 0 0 0 1 2 0 0 18 0 1 0 616961392 308502528 6020 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198790 0 0 16781312 514 16926972713518465288 0 0 17 1 0 0 0
/proc/11053/statm: 75318 6020 372 364 0 11230 0
[pid=11055] ppid=11053 vsize=420220 CPUtime=32.58 cores=1,3,5,7
/proc/11055/stat : 11055 (adaptg2wsat2009) R 11053 11055 11055 0 -1 4202496 1668 0 0 0 3258 0 0 0 25 0 1 0 616961392 430305280 1400 33554432000 134512640 135107411 4288500720 18446744073709551615 134531674 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/11055/statm: 105055 1400 59 146 0 103931 0
[pid=11056] ppid=11023 vsize=312136 CPUtime=0.03 cores=1,3,5,7
/proc/11056/stat : 11056 (python) S 11023 11023 10986 0 -1 4202560 939 0 0 0 2 1 0 0 18 0 1 0 616961395 319627264 6024 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198790 0 0 16781312 514 16926972713518465288 0 0 17 7 0 0 0
/proc/11056/statm: 78034 6024 372 364 0 13943 0
[pid=11058] ppid=11056 vsize=420220 CPUtime=32.33 cores=1,3,5,7
/proc/11058/stat : 11058 (adaptg2wsat2009) R 11056 11058 11058 0 -1 4202496 1662 0 0 0 3233 0 0 0 25 0 1 0 616961395 430305280 1399 33554432000 134512640 135107411 4288970480 18446744073709551615 134532135 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/11058/statm: 105055 1399 59 146 0 103931 0
[pid=11023/tid=11048] ppid=11019 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11048/stat : 11048 (python) S 11019 11023 10986 0 -1 4202560 23 927 0 8 0 0 0 0 19 0 5 0 616961386 328998912 7538 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 1 0 0 0
[pid=11023/tid=11051] ppid=11019 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11051/stat : 11051 (python) S 11019 11023 10986 0 -1 4202560 23 927 0 8 0 0 0 0 20 0 5 0 616961389 328998912 7538 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11054] ppid=11019 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11054/stat : 11054 (python) S 11019 11023 10986 0 -1 4202560 22 927 0 8 0 0 0 0 22 0 5 0 616961392 328998912 7538 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11057] ppid=11019 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11057/stat : 11057 (python) S 11019 11023 10986 0 -1 4202560 25 927 0 8 0 0 0 0 24 0 5 0 616961395 328998912 7538 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 132.48
Current children cumulated vsize (KiB) 4386396

[startup+102.301 s]
/proc/loadavg: 6.25 2.07 0.73 10/202 11060
/proc/meminfo: memFree=23163124/32951124 swapFree=67102140/67111528
[pid=11023] ppid=11019 vsize=321288 CPUtime=2.16 cores=1,3,5,7
/proc/11023/stat : 11023 (python) S 11019 11023 10986 0 -1 4202496 15277 927 114 8 198 18 0 0 24 0 5 0 616959545 328998912 7538 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 310
/proc/11023/statm: 80322 7538 1883 364 0 16231 0
[pid=11047] ppid=11023 vsize=280760 CPUtime=0.1 cores=1,3,5,7
/proc/11047/stat : 11047 (python) S 11023 11023 10986 0 -1 4202560 914 0 0 0 6 4 0 0 18 0 1 0 616961386 287498240 6001 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198735 0 0 16781312 514 16926972713518465288 0 0 17 3 0 0 0
/proc/11047/statm: 70190 6001 378 364 0 6108 0
[pid=11049] ppid=11047 vsize=1019708 CPUtime=83.88 cores=1,3,5,7
/proc/11049/stat : 11049 (gnovelty+2) R 11047 11049 11049 0 -1 4202496 955 0 0 0 8388 0 0 0 25 0 1 0 616961386 1044180992 690 33554432000 134512640 135663704 4289265600 18446744073709551615 134524901 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/11049/statm: 254927 690 113 282 0 254643 0
[pid=11050] ppid=11023 vsize=291084 CPUtime=0.1 cores=1,3,5,7
/proc/11050/stat : 11050 (python) S 11023 11023 10986 0 -1 4202560 923 0 0 0 5 5 0 0 18 0 1 0 616961389 298070016 5999 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198790 0 0 16781312 514 16926972713518465288 0 0 17 7 0 0 0
/proc/11050/statm: 72771 5999 372 364 0 8686 0

################
# More data... #
################

/proc/11055/stat : 11055 (adaptg2wsat2009) T 11053 11055 11055 0 -1 4202496 1668 0 0 0 10000 0 0 0 25 0 1 0 616961392 430305280 1400 33554432000 134512640 135107411 4288500720 18446744073709551615 134531411 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/11055/statm: 105055 1400 59 146 0 103931 0
[pid=11053/tid=11089] ppid=11023 vsize=301272 CPUtime=0 cores=1,3,5,7
/proc/11053/task/11089/stat : 11089 (python) S 11023 11023 10986 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 616971406 308502528 6049 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604459687811776 0 0 -1 1 0 0 0
[pid=11056] ppid=11023 vsize=312136 CPUtime=0.13 cores=1,3,5,7
/proc/11056/stat : 11056 (python) S 11023 11023 10986 0 -1 4202560 1116 0 0 0 6 7 0 0 19 0 2 0 616961395 319627264 6053 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/11056/statm: 78034 6053 400 364 0 13943 0
[pid=11058] ppid=11056 vsize=420220 CPUtime=98.34 cores=1,3,5,7
/proc/11058/stat : 11058 (adaptg2wsat2009) T 11056 11058 11058 0 -1 4202496 1662 0 0 0 9834 0 0 0 25 0 1 0 616961395 430305280 1399 33554432000 134512640 135107411 4288970480 18446744073709551615 134531615 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/11058/statm: 105055 1399 59 146 0 103931 0
[pid=11056/tid=11064] ppid=11023 vsize=312136 CPUtime=0 cores=1,3,5,7
/proc/11056/task/11064/stat : 11064 (python) S 11023 11023 10986 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 616971310 319627264 6053 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604471128781056 0 0 -1 7 0 0 0
[pid=11062] ppid=11023 vsize=322004 CPUtime=0.13 cores=1,3,5,7
/proc/11062/stat : 11062 (python) S 11023 11023 10986 0 -1 4202560 1131 0 0 0 8 5 0 0 19 0 2 0 616971309 329732096 6057 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/11062/statm: 80501 6057 400 364 0 16407 0
[pid=11065] ppid=11062 vsize=420220 CPUtime=100.11 cores=1,3,5,7
/proc/11065/stat : 11065 (adaptg2wsat2009) T 11062 11065 11065 0 -1 4202496 1668 0 0 0 10011 0 0 0 25 0 1 0 616971310 430305280 1400 33554432000 134512640 135107411 4292392672 18446744073709551615 134532349 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/11065/statm: 105055 1400 59 146 0 103931 0
[pid=11062/tid=11140] ppid=11023 vsize=322004 CPUtime=0 cores=1,3,5,7
/proc/11062/task/11140/stat : 11140 (python) S 11023 11023 10986 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 616981322 329732096 6057 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604466106737984 0 0 -1 3 0 0 0
[pid=11066] ppid=11023 vsize=332124 CPUtime=0.13 cores=1,3,5,7
/proc/11066/stat : 11066 (python) S 11023 11023 10986 0 -1 4202560 1128 0 0 0 8 5 0 0 19 0 2 0 616971313 340094976 6099 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/11066/statm: 83031 6099 400 364 0 18934 0
[pid=11068] ppid=11066 vsize=1019708 CPUtime=98.39 cores=1,3,5,7
/proc/11068/stat : 11068 (gnovelty+2) T 11066 11068 11068 0 -1 4202496 953 0 0 0 9839 0 0 0 25 0 1 0 616971313 1044180992 690 33554432000 134512640 135663704 4291826512 18446744073709551615 134545631 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/11068/statm: 254927 690 113 282 0 254643 0
[pid=11066/tid=11145] ppid=11023 vsize=332124 CPUtime=0 cores=1,3,5,7
/proc/11066/task/11145/stat : 11145 (python) S 11023 11023 10986 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 616981330 340094976 6099 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604466106738880 0 0 -1 7 0 0 0
[pid=11086] ppid=11023 vsize=342560 CPUtime=0.13 cores=1,3,5,7
/proc/11086/stat : 11086 (python) S 11023 11023 10986 0 -1 4202560 1171 0 0 0 7 6 0 0 19 0 2 0 616971405 350781440 6109 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/11086/statm: 85640 6109 406 364 0 21540 0
[pid=11088] ppid=11086 vsize=249872 CPUtime=99.97 cores=1,3,5,7
/proc/11088/stat : 11088 (FH) T 11086 11088 11088 0 -1 4202496 1532 0 0 0 9997 0 0 0 25 0 1 0 616971406 255868928 1264 33554432000 134512640 134936988 4292220560 18446744073709551615 134553238 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/11088/statm: 62468 1264 41 104 0 61972 0
[pid=11086/tid=11152] ppid=11023 vsize=342560 CPUtime=0 cores=1,3,5,7
/proc/11086/task/11152/stat : 11152 (python) S 11023 11023 10986 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 616981418 350781440 6109 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604471124425472 0 0 -1 1 0 0 0
[pid=11090] ppid=11023 vsize=352652 CPUtime=0.13 cores=1,3,5,7
/proc/11090/stat : 11090 (python) S 11023 11023 10986 0 -1 4202560 1176 0 0 0 8 5 0 0 19 0 2 0 616971409 361115648 6124 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/11090/statm: 88163 6124 406 364 0 24060 0
[pid=11092] ppid=11090 vsize=420220 CPUtime=98.99 cores=1,3,5,7
/proc/11092/stat : 11092 (adaptg2wsat2009) T 11090 11092 11092 0 -1 4202496 1673 0 0 0 9899 0 0 0 25 0 1 0 616971410 430305280 1401 33554432000 134512640 135107411 4291113440 18446744073709551615 134532204 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/11092/statm: 105055 1401 59 146 0 103931 0
[pid=11090/tid=11141] ppid=11023 vsize=352652 CPUtime=0 cores=1,3,5,7
/proc/11090/task/11141/stat : 11141 (python) S 11023 11023 10986 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 616981322 361115648 6124 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604471143335808 0 0 -1 7 0 0 0
[pid=11142] ppid=11023 vsize=363512 CPUtime=0.14 cores=1,3,5,7
/proc/11142/stat : 11142 (python) S 11023 11023 10986 0 -1 4202560 1151 0 0 0 8 6 0 0 19 0 2 0 616981326 372236288 6122 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/11142/statm: 90878 6122 400 364 0 26772 0
[pid=11144] ppid=11142 vsize=420220 CPUtime=99.98 cores=1,3,5,7
/proc/11144/stat : 11144 (adaptg2wsat2009) T 11142 11144 11144 0 -1 4202496 1669 0 0 0 9998 0 0 0 25 0 1 0 616981327 430305280 1400 33554432000 134512640 135107411 4289240800 18446744073709551615 134539292 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/11144/statm: 105055 1400 59 146 0 103931 0
[pid=11142/tid=11174] ppid=11023 vsize=363512 CPUtime=0 cores=1,3,5,7
/proc/11142/task/11174/stat : 11174 (python) S 11023 11023 10986 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 616991340 372236288 6122 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604471131135040 0 0 -1 5 0 0 0
[pid=11146] ppid=11023 vsize=373484 CPUtime=0.15 cores=1,3,5,7
/proc/11146/stat : 11146 (python) S 11023 11023 10986 0 -1 4202560 1163 0 0 0 8 7 0 0 19 0 2 0 616981331 382447616 6236 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/11146/statm: 93371 6236 400 364 0 29262 0
[pid=11148] ppid=11146 vsize=249876 CPUtime=100.13 cores=1,3,5,7
/proc/11148/stat : 11148 (FH) T 11146 11148 11148 0 -1 4202496 1532 0 0 0 10013 0 0 0 25 0 1 0 616981331 255873024 1264 33554432000 134512640 134936988 4292286224 18446744073709551615 134545109 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/11148/statm: 62469 1264 41 104 0 61973 0
[pid=11146/tid=11178] ppid=11023 vsize=373484 CPUtime=0 cores=1,3,5,7
/proc/11146/task/11178/stat : 11178 (python) S 11023 11023 10986 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 616991345 382447616 6236 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604471143336704 0 0 -1 1 0 0 0
[pid=11149] ppid=11023 vsize=383740 CPUtime=0.62 cores=1,3,5,7
/proc/11149/stat : 11149 (python) S 11023 11023 10986 0 -1 4202560 1191 0 0 0 32 30 0 0 18 0 2 0 616981335 392949760 6240 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/11149/statm: 95935 6240 400 364 0 31823 0
[pid=11151] ppid=11149 vsize=268420 CPUtime=389.39 cores=1,3,5,7
/proc/11151/stat : 11151 (hybridGM3) T 11149 11151 11151 0 -1 4202496 5396 0 0 0 38939 0 0 0 25 0 1 0 616981336 274862080 5126 33554432000 134512640 135204385 4291625728 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/11151/statm: 67105 5126 63 169 0 66935 0
[pid=11149/tid=11209] ppid=11023 vsize=383740 CPUtime=0 cores=1,3,5,7
/proc/11149/task/11209/stat : 11209 (python) S 11023 11023 10986 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 617001471 392949760 6240 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604456345955200 0 0 -1 7 0 0 0
[pid=11153] ppid=11023 vsize=393796 CPUtime=0.14 cores=1,3,5,7
/proc/11153/stat : 11153 (python) S 11023 11023 10986 0 -1 4202560 1173 0 0 0 8 6 0 0 19 0 2 0 616981422 403247104 6243 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/11153/statm: 98449 6243 399 364 0 34334 0
[pid=11155] ppid=11153 vsize=1019708 CPUtime=100.02 cores=1,3,5,7
/proc/11155/stat : 11155 (gnovelty+2) T 11153 11155 11155 0 -1 4202496 961 0 0 0 10002 0 0 0 25 0 1 0 616981422 1044180992 690 33554432000 134512640 135663704 4288725008 18446744073709551615 134525255 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/11155/statm: 254927 690 113 282 0 254643 0
[pid=11153/tid=11183] ppid=11023 vsize=393796 CPUtime=0 cores=1,3,5,7
/proc/11153/task/11183/stat : 11183 (python) S 11023 11023 10986 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 616991440 403247104 6243 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604456345953408 0 0 -1 7 0 0 0
[pid=11175] ppid=11023 vsize=404712 CPUtime=0.46 cores=1,3,5,7
/proc/11175/stat : 11175 (python) S 11023 11023 10986 0 -1 4202560 1189 0 0 0 26 20 0 0 18 0 2 0 616991345 414425088 6248 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198790 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/11175/statm: 101178 6248 400 364 0 37060 0
[pid=11177] ppid=11175 vsize=268420 CPUtime=291.56 cores=1,3,5,7
/proc/11177/stat : 11177 (hybridGM3) R 11175 11177 11177 0 -1 4202496 5395 0 0 0 29155 1 0 0 25 0 1 0 616991345 274862080 5125 33554432000 134512640 135204385 4289642288 18446744073709551615 134538435 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/11177/statm: 67105 5125 63 169 0 66935 0
[pid=11175/tid=11225] ppid=11023 vsize=404712 CPUtime=0 cores=1,3,5,7
/proc/11175/task/11225/stat : 11225 (python) S 11023 11023 10986 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 617010981 414425088 6248 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604471143334016 0 0 -1 7 0 0 0
[pid=11179] ppid=11023 vsize=414772 CPUtime=0.14 cores=1,3,5,7
/proc/11179/stat : 11179 (python) S 11023 11023 10986 0 -1 4202560 1184 0 0 0 8 6 0 0 19 0 2 0 616991350 424726528 6296 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/11179/statm: 103693 6296 400 364 0 39572 0
[pid=11181] ppid=11179 vsize=420220 CPUtime=99.83 cores=1,3,5,7
/proc/11181/stat : 11181 (adaptg2wsat2009) T 11179 11181 11181 0 -1 4202496 1671 0 0 0 9983 0 0 0 25 0 1 0 616991350 430305280 1399 33554432000 134512640 135107411 4288200112 18446744073709551615 134532376 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/11181/statm: 105055 1399 59 146 0 103931 0
[pid=11179/tid=11205] ppid=11023 vsize=414772 CPUtime=0 cores=1,3,5,7
/proc/11179/task/11205/stat : 11205 (python) S 11023 11023 10986 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 617001364 424726528 6296 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604468433845184 0 0 -1 1 0 0 0
[pid=11184] ppid=11023 vsize=424920 CPUtime=0.46 cores=1,3,5,7
/proc/11184/stat : 11184 (python) S 11023 11023 10986 0 -1 4202560 1244 0 0 0 28 18 0 0 15 0 2 0 616991445 435118080 6307 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/11184/statm: 106230 6307 407 364 0 42106 0
[pid=11186] ppid=11184 vsize=268428 CPUtime=291.17 cores=1,3,5,7
/proc/11186/stat : 11186 (hybridGM3) T 11184 11186 11186 0 -1 4202496 5400 0 0 0 29117 0 0 0 25 0 1 0 616991446 274870272 5127 33554432000 134512640 135204385 4294879120 18446744073709551615 134537704 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/11186/statm: 67107 5127 63 169 0 66937 0
[pid=11184/tid=11226] ppid=11023 vsize=424920 CPUtime=0 cores=1,3,5,7
/proc/11184/task/11226/stat : 11226 (python) S 11023 11023 10986 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 617011081 435118080 6307 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 514 18446604468433843392 0 0 -1 1 0 0 0
[pid=11206] ppid=11023 vsize=435332 CPUtime=0.31 cores=1,3,5,7
/proc/11206/stat : 11206 (python) S 11023 11023 10986 0 -1 4202560 1075 0 0 0 17 14 0 0 18 0 1 0 617001369 445779968 6283 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198790 0 0 16781312 514 16926972713518465284 0 0 17 1 0 0 0
/proc/11206/statm: 108833 6283 380 364 0 44706 0
[pid=11208] ppid=11206 vsize=268420 CPUtime=187.67 cores=1,3,5,7
/proc/11208/stat : 11208 (hybridGM3) R 11206 11208 11208 0 -1 4202496 5398 0 0 0 18766 1 0 0 25 0 1 0 617001369 274862080 5125 33554432000 134512640 135204385 4288294672 18446744073709551615 134537717 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/11208/statm: 67105 5125 63 169 0 66935 0
[pid=11233] ppid=11023 vsize=445932 CPUtime=0 cores=1,3,5,7
/proc/11233/stat : 11233 (python) S 11023 11023 10986 0 -1 4202560 851 0 0 0 0 0 0 0 18 0 1 0 617020514 456634368 6258 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198790 0 0 16781312 514 14633045116109979647 0 0 17 1 0 0 0
/proc/11233/statm: 111483 6258 351 364 0 47353 0
[pid=11235] ppid=11233 vsize=454088 CPUtime=1.9 cores=1,3,5,7
/proc/11235/stat : 11235 (TNM) R 11233 11235 11235 0 -1 4202496 1274 0 0 0 190 0 0 0 25 0 1 0 617020515 464986112 995 33554432000 134512640 135008748 4289212112 18446744073709551615 134523379 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/11235/statm: 113522 995 45 122 0 112422 0
[pid=11236] ppid=11023 vsize=456096 CPUtime=0 cores=1,3,5,7
/proc/11236/stat : 11236 (python) S 11023 11023 10986 0 -1 4202560 839 0 0 0 0 0 0 0 18 0 1 0 617020605 467042304 6261 33554432000 4194304 5683644 140734223175792 18446744073709551615 216735198790 0 0 16781312 514 14633041267819282431 0 0 17 1 0 0 0
/proc/11236/statm: 114024 6261 350 364 0 49891 0
[pid=11238] ppid=11236 vsize=247956 CPUtime=1.01 cores=1,3,5,7
/proc/11238/stat : 11238 (FH) R 11236 11238 11238 0 -1 4202496 1257 0 0 0 101 0 0 0 24 0 1 0 617020605 253906944 986 33554432000 134512640 134936988 4292177728 18446744073709551615 134523547 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/11238/statm: 61989 986 37 104 0 61493 0
[pid=11023/tid=11048] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11048/stat : 11048 (python) S 11019 11023 10986 0 -1 4202560 23 927 0 8 0 0 0 0 19 0 19 0 616961386 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 1 0 0 0
[pid=11023/tid=11051] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11051/stat : 11051 (python) S 11019 11023 10986 0 -1 4202560 23 927 0 8 0 0 0 0 20 0 19 0 616961389 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11054] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11054/stat : 11054 (python) S 11019 11023 10986 0 -1 4202560 22 927 0 8 0 0 0 0 22 0 19 0 616961392 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11057] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11057/stat : 11057 (python) S 11019 11023 10986 0 -1 4202560 25 927 0 8 0 0 0 0 24 0 19 0 616961395 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11063] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11063/stat : 11063 (python) S 11019 11023 10986 0 -1 4202560 20 927 0 8 0 0 0 0 20 0 19 0 616971309 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 3 0 0 0
[pid=11023/tid=11067] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11067/stat : 11067 (python) S 11019 11023 10986 0 -1 4202560 30 927 0 8 0 0 0 0 21 0 19 0 616971313 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11087] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11087/stat : 11087 (python) S 11019 11023 10986 0 -1 4202560 26 927 0 8 0 0 0 0 20 0 19 0 616971405 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11091] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11091/stat : 11091 (python) S 11019 11023 10986 0 -1 4202560 30 927 0 8 0 0 0 0 22 0 19 0 616971409 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11143] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11143/stat : 11143 (python) S 11019 11023 10986 0 -1 4202560 24 927 0 8 0 0 0 0 20 0 19 0 616981326 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 5 0 0 0
[pid=11023/tid=11147] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11147/stat : 11147 (python) S 11019 11023 10986 0 -1 4202560 22 927 0 8 0 0 0 0 22 0 19 0 616981331 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 1 0 0 0
[pid=11023/tid=11150] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11150/stat : 11150 (python) S 11019 11023 10986 0 -1 4202560 34 927 0 8 0 0 0 0 18 0 19 0 616981336 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11154] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11154/stat : 11154 (python) S 11019 11023 10986 0 -1 4202560 31 927 0 8 0 0 0 0 20 0 19 0 616981422 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11176] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11176/stat : 11176 (python) S 11019 11023 10986 0 -1 4202560 32 927 0 8 0 0 0 0 18 0 19 0 616991345 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 7 0 0 0
[pid=11023/tid=11180] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11180/stat : 11180 (python) S 11019 11023 10986 0 -1 4202560 25 927 0 8 0 0 0 0 21 0 19 0 616991350 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 1 0 0 0
[pid=11023/tid=11185] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11185/stat : 11185 (python) S 11019 11023 10986 0 -1 4202560 26 927 0 8 0 0 0 0 18 0 19 0 616991446 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 1 0 0 0
[pid=11023/tid=11207] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11207/stat : 11207 (python) S 11019 11023 10986 0 -1 4202560 27 927 0 8 0 0 0 0 20 0 19 0 617001369 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 1 0 0 0
[pid=11023/tid=11234] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11234/stat : 11234 (python) S 11019 11023 10986 0 -1 4202560 25 927 0 8 0 0 0 0 20 0 19 0 617020514 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 1 0 0 0
[pid=11023/tid=11237] ppid=11019 vsize=465752 CPUtime=0 cores=1,3,5,7
/proc/11023/task/11237/stat : 11237 (python) S 11019 11023 10986 0 -1 4202560 27 927 0 8 0 0 0 0 20 0 19 0 617020605 476930048 7826 33554432000 4194304 5683644 140734223175792 18446744073709551615 216747002289 0 0 16781312 2 18446604454224049216 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2364.02
Current children cumulated vsize (KiB) 15982516

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 11023 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=759580
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=235964
# CPU time returned by wait4() is 2.99554
# while last known CPU time is 2364.02
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 611.719
CPU time (s): 2364.02
CPU user time (s): 2362.26
CPU system time (s): 1.76
CPU usage (%): 386.455
Max. virtual memory (cumulated for all children) (KiB): 15982516

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.75958
system time used= 0.235964
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 30022
page faults= 122
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 17379
involuntary context switches= 141

runsolver used 3.66444 second user time and 6.85596 second system time

The end

Launcher Data 

Begin job on node119 at 2011-05-01 18:28:05
IDJOB=3317077
IDBENCH=82542
IDSOLVER=1705
FILE ID=node119/3317077-1304267285
RUNJOBID= node119-1304267283-11005
PBS_JOBID= 13173960
Free space on /tmp= 73052 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k7-r85-v300-c25500-S942666749-061-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317077-1304267285/watcher-3317077-1304267285 -o /tmp/evaluation-result-3317077-1304267285/solver-3317077-1304267285 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317077-1304267285.cnf -seed 40564498 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 456ae67a44e812130272597099547869
RANDOM SEED=40564498

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23305472 kB
Buffers:       1963564 kB
Cached:        7044512 kB
SwapCached:       3688 kB
Active:         624168 kB
Inactive:      8443284 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23305472 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:           64916 kB
Writeback:           0 kB
AnonPages:       58660 kB
Mapped:          14396 kB
Slab:           514664 kB
PageTables:       4668 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   188680 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72992 MiB
End job on node119 at 2011-05-01 18:38:23