Trace number 3317081

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 1979.32 513.73

General information on the benchmark

Namerandom/large/
unif-k7-r85-v300-c25500-S1286362348-008-UNKNOWN.cnf
MD5SUM63b66954174098a654eacd27832fcf38
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables300
Number of clauses25500
Sum of the clauses size178500
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 525500

Solver Data 

0.67/15.64	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.67/15.64	c INFO: solving HOME/instance-3317081-1304267286.cnf
0.67/15.64	c INFO: computing task features
0.84/15.81	c INFO: computed variable-clause graph statistics
0.84/15.82	c INFO: computed clause balance statistics
1.74/16.75	c INFO: computed variable balance statistics
2.14/17.12	c INFO: computed small-clause counts
2.14/17.12	c INFO: computed Horn-clause counts
2.14/17.15	c INFO: computed variable graph statistics
2.14/17.15	c INFO: collected features for HOME/instance-3317081-1304267286.cnf in 1.51 s
2.20/17.24	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.37)
2.20/17.27	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.13)
2.27/17.30	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.08)
2.27/17.33	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.07)
398.04/116.46	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.05)
398.04/116.50	c INFO: running gnovelty+2@0 for 97 with 4699 remaining (b = 0.04)
398.42/116.54	c INFO: running FH@0 for 97 with 4699 remaining (b = 0.03)
400.24/117.47	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.03)
797.28/216.64	c INFO: running adaptg2wsat2009++@0 for 97 with 4599 remaining (b = 0.02)
797.28/216.68	c INFO: running FH@0 for 97 with 4598 remaining (b = 0.02)
797.56/216.73	c INFO: running hybridGM3@0 for 194 with 4598 remaining (b = 0.04)
797.56/216.77	c INFO: running gnovelty+2@0 for 97 with 4598 remaining (b = 0.02)
1196.33/316.83	c INFO: running hybridGM3@0 for 194 with 4498 remaining (b = 0.03)
1196.33/316.87	c INFO: running adaptg2wsat2009++@0 for 97 with 4498 remaining (b = 0.01)
1196.81/317.00	c INFO: running hybridGM3@0 for 194 with 4498 remaining (b = 0.03)
1579.82/413.11	c INFO: running hybridGM3@100 for 97 with 4402 remaining (b = 0.01)
1595.35/417.06	c INFO: running hybridGM3@0 for 194 with 4398 remaining (b = 0.02)
1957.88/508.28	c INFO: running TNM@0 for 1552 with 4307 remaining (b = 0.19)
1978.09/513.39	c INFO: running FH@0 for 97 with 4302 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317081-1304267286/watcher-3317081-1304267286 -o /tmp/evaluation-result-3317081-1304267286/solver-3317081-1304267286 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317081-1304267286.cnf -seed 1878399543 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.16 0.03 0.01 2/176 12811
/proc/meminfo: memFree=29517236/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=1840 CPUtime=0 cores=1,3,5,7
/proc/12809/stat : 12809 (python) D 12807 12809 12773 0 -1 4194304 47 0 0 0 0 0 0 0 20 0 1 0 616959763 1884160 1 33554432000 0 0 140733432931993 18446744073709551615 5055495 0 0 4096 0 18446744071562234166 0 0 17 1 0 0 0
/proc/12809/statm: 460 1 0 0 0 36 0

[startup+0.073499 s]
/proc/loadavg: 0.16 0.03 0.01 2/176 12811
/proc/meminfo: memFree=29517236/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=15084 CPUtime=0 cores=1,3,5,7
/proc/12809/stat : 12809 (python) S 12807 12809 12773 0 -1 4202496 614 0 6 0 0 0 0 0 17 0 1 0 616959763 15446016 516 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415387572 0 2147483391 16781312 2 18446744071699945619 0 0 17 1 0 0 2
/proc/12809/statm: 3771 516 302 364 0 258 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15084

[startup+0.1005 s]
/proc/loadavg: 0.16 0.03 0.01 2/176 12811
/proc/meminfo: memFree=29517236/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=15184 CPUtime=0 cores=1,3,5,7
/proc/12809/stat : 12809 (python) S 12807 12809 12773 0 -1 4202496 718 0 6 0 0 0 0 0 17 0 1 0 616959763 15548416 614 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415389696 0 2147483391 16781312 2 18446744071699199166 0 0 17 1 0 0 3
/proc/12809/statm: 3796 614 321 364 0 283 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15184

[startup+0.300458 s]
/proc/loadavg: 0.16 0.03 0.01 2/176 12811
/proc/meminfo: memFree=29517236/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=15916 CPUtime=0 cores=1,3,5,7
/proc/12809/stat : 12809 (python) S 12807 12809 12773 0 -1 4202496 954 0 6 0 0 0 0 0 15 0 1 0 616959763 16297984 835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415387572 0 2147483391 16781312 2 18446744071699199166 0 0 17 1 0 0 4
/proc/12809/statm: 3979 835 357 364 0 466 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15916

[startup+0.700372 s]
/proc/loadavg: 0.16 0.03 0.01 2/176 12811
/proc/meminfo: memFree=29517236/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=23188 CPUtime=0.04 cores=1,3,5,7
/proc/12809/stat : 12809 (python) D 12807 12809 12773 0 -1 4202496 1315 0 9 0 2 2 0 0 15 0 1 0 616959763 23744512 1173 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415389696 0 0 16781312 2 0 0 0 17 1 0 0 6
/proc/12809/statm: 5797 1173 432 364 0 724 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 23188

[startup+1.50023 s]
/proc/loadavg: 0.16 0.03 0.01 2/178 12813
/proc/meminfo: memFree=29506144/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=40976 CPUtime=0.08 cores=1,3,5,7
/proc/12809/stat : 12809 (python) R 12807 12809 12773 0 -1 4202496 1663 0 15 0 4 4 0 0 16 0 1 0 616959763 41959424 1502 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415389696 0 0 16781312 2 0 0 0 17 1 0 0 9
/proc/12809/statm: 10244 1502 463 364 0 1027 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 40976

[startup+3.10091 s]
/proc/loadavg: 0.23 0.05 0.02 1/178 12813
/proc/meminfo: memFree=29500420/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=94216 CPUtime=0.12 cores=1,3,5,7
/proc/12809/stat : 12809 (python) D 12807 12809 12773 0 -1 4202496 2756 0 23 0 5 7 0 0 15 0 1 0 616959763 96477184 2484 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415389696 0 0 16781312 2 0 0 0 17 3 0 0 16
/proc/12809/statm: 23554 2484 765 364 0 1686 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 94216
heavy processes:

[startup+6.30027 s]
/proc/loadavg: 0.23 0.05 0.02 1/178 12813
/proc/meminfo: memFree=29489156/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=105100 CPUtime=0.18 cores=1,3,5,7
/proc/12809/stat : 12809 (python) D 12807 12809 12773 0 -1 4202496 3226 0 31 0 7 11 0 0 16 0 1 0 616959763 107622400 2922 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415389696 0 0 16781312 2 0 0 0 17 7 0 0 120
/proc/12809/statm: 26275 2922 868 364 0 2023 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 105100
heavy processes:

[startup+12.701 s]
/proc/loadavg: 0.35 0.08 0.03 1/178 12821
/proc/meminfo: memFree=29471568/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=138876 CPUtime=0.43 cores=1,3,5,7
/proc/12809/stat : 12809 (python) S 12807 12809 12773 0 -1 4202496 6031 928 43 8 20 23 0 0 16 0 1 0 616959763 142209024 4207 33554432000 4194304 5683644 140733432923968 18446744073709551615 256410467975 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 258
/proc/12809/statm: 34719 4207 1034 364 0 3161 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 138876
heavy processes:

[startup+25.5006 s]
/proc/loadavg: 1.08 0.24 0.08 10/202 12845
/proc/meminfo: memFree=29293020/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=321292 CPUtime=2.3 cores=1,3,5,7
/proc/12809/stat : 12809 (python) S 12807 12809 12773 0 -1 4202496 15282 928 114 8 196 34 0 0 22 0 5 0 616959763 329003008 7538 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 361
/proc/12809/statm: 80323 7538 1883 364 0 16232 0
[pid=12822] ppid=12809 vsize=280764 CPUtime=0 cores=1,3,5,7
/proc/12822/stat : 12822 (python) S 12809 12809 12773 0 -1 4202560 884 0 0 0 0 0 0 0 18 0 1 0 616961488 287502336 6001 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411727 0 0 16781312 514 4381247077238145288 0 0 17 7 0 0 0
/proc/12822/statm: 70191 6001 378 364 0 6109 0
[pid=12824] ppid=12822 vsize=1019708 CPUtime=8.17 cores=1,3,5,7
/proc/12824/stat : 12824 (gnovelty+2) R 12822 12824 12824 0 -1 4202496 954 0 0 0 817 0 0 0 25 0 1 0 616961489 1044180992 689 33554432000 134512640 135663704 4293165872 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12824/statm: 254927 689 113 282 0 254643 0
[pid=12825] ppid=12809 vsize=291088 CPUtime=0 cores=1,3,5,7
/proc/12825/stat : 12825 (python) S 12809 12809 12773 0 -1 4202560 894 0 0 0 0 0 0 0 18 0 1 0 616961491 298074112 5999 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411782 0 0 16781312 514 17631206067113263368 0 0 17 5 0 0 0
/proc/12825/statm: 72772 5999 372 364 0 8687 0
[pid=12827] ppid=12825 vsize=1019708 CPUtime=8.21 cores=1,3,5,7
/proc/12827/stat : 12827 (gnovelty+2) R 12825 12827 12827 0 -1 4202496 954 0 0 0 821 0 0 0 25 0 1 0 616961491 1044180992 690 33554432000 134512640 135663704 4294642000 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/12827/statm: 254927 690 113 282 0 254643 0
[pid=12828] ppid=12809 vsize=301276 CPUtime=0 cores=1,3,5,7
/proc/12828/stat : 12828 (python) S 12809 12809 12773 0 -1 4202560 896 0 0 0 0 0 0 0 18 0 1 0 616961494 308506624 6019 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411782 0 0 16781312 514 4445328539050279176 0 0 17 7 0 0 0
/proc/12828/statm: 75319 6019 371 364 0 11231 0
[pid=12830] ppid=12828 vsize=420220 CPUtime=8.18 cores=1,3,5,7
/proc/12830/stat : 12830 (adaptg2wsat2009) R 12828 12830 12830 0 -1 4202496 1669 0 0 0 818 0 0 0 25 0 1 0 616961495 430305280 1400 33554432000 134512640 135107411 4290924544 18446744073709551615 134532121 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12830/statm: 105055 1400 59 146 0 103931 0
[pid=12834] ppid=12809 vsize=312140 CPUtime=0 cores=1,3,5,7
/proc/12834/stat : 12834 (python) S 12809 12809 12773 0 -1 4202560 922 0 0 0 0 0 0 0 18 0 1 0 616961497 319631360 6024 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411782 0 0 16781312 514 9078614463405326600 0 0 17 5 0 0 0
/proc/12834/statm: 78035 6024 372 364 0 13944 0
[pid=12836] ppid=12834 vsize=420220 CPUtime=8.14 cores=1,3,5,7
/proc/12836/stat : 12836 (adaptg2wsat2009) R 12834 12836 12836 0 -1 4202496 1665 0 0 0 814 0 0 0 25 0 1 0 616961497 430305280 1400 33554432000 134512640 135107411 4288013792 18446744073709551615 134531535 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12836/statm: 105055 1400 59 146 0 103931 0
[pid=12809/tid=12823] ppid=12807 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12823/stat : 12823 (python) S 12807 12809 12773 0 -1 4202560 25 928 0 8 0 0 0 0 19 0 5 0 616961488 329003008 7538 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
[pid=12809/tid=12826] ppid=12807 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12826/stat : 12826 (python) S 12807 12809 12773 0 -1 4202560 21 928 0 8 0 0 0 0 21 0 5 0 616961491 329003008 7538 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
[pid=12809/tid=12829] ppid=12807 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12829/stat : 12829 (python) S 12807 12809 12773 0 -1 4202560 21 928 0 8 0 0 0 0 20 0 5 0 616961494 329003008 7538 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
[pid=12809/tid=12835] ppid=12807 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12835/stat : 12835 (python) S 12807 12809 12773 0 -1 4202560 27 928 0 8 0 0 0 0 22 0 5 0 616961497 329003008 7538 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 35
Current children cumulated vsize (KiB) 4386416

[startup+51.1007 s]
/proc/loadavg: 3.44 0.86 0.29 10/202 12846
/proc/meminfo: memFree=29292512/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=321292 CPUtime=2.3 cores=1,3,5,7
/proc/12809/stat : 12809 (python) S 12807 12809 12773 0 -1 4202496 15282 928 114 8 196 34 0 0 22 0 5 0 616959763 329003008 7538 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 361
/proc/12809/statm: 80323 7538 1883 364 0 16232 0
[pid=12822] ppid=12809 vsize=280764 CPUtime=0.04 cores=1,3,5,7
/proc/12822/stat : 12822 (python) S 12809 12809 12773 0 -1 4202560 895 0 0 0 2 2 0 0 18 0 1 0 616961488 287502336 6001 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411727 0 0 16781312 514 18224044590573256968 0 0 17 7 0 0 0
/proc/12822/statm: 70191 6001 378 364 0 6109 0
[pid=12824] ppid=12822 vsize=1019708 CPUtime=33.53 cores=1,3,5,7
/proc/12824/stat : 12824 (gnovelty+2) R 12822 12824 12824 0 -1 4202496 954 0 0 0 3353 0 0 0 25 0 1 0 616961489 1044180992 689 33554432000 134512640 135663704 4293165872 18446744073709551615 134525544 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12824/statm: 254927 689 113 282 0 254643 0
[pid=12825] ppid=12809 vsize=291088 CPUtime=0.03 cores=1,3,5,7
/proc/12825/stat : 12825 (python) S 12809 12809 12773 0 -1 4202560 905 0 0 0 2 1 0 0 18 0 1 0 616961491 298074112 5999 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411782 0 0 16781312 514 18224044590573256968 0 0 17 5 0 0 0
/proc/12825/statm: 72772 5999 372 364 0 8687 0
[pid=12827] ppid=12825 vsize=1019708 CPUtime=33.81 cores=1,3,5,7
/proc/12827/stat : 12827 (gnovelty+2) R 12825 12827 12827 0 -1 4202496 954 0 0 0 3381 0 0 0 25 0 1 0 616961491 1044180992 690 33554432000 134512640 135663704 4294642000 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/12827/statm: 254927 690 113 282 0 254643 0
[pid=12828] ppid=12809 vsize=301276 CPUtime=0.04 cores=1,3,5,7
/proc/12828/stat : 12828 (python) S 12809 12809 12773 0 -1 4202560 907 0 0 0 2 2 0 0 18 0 1 0 616961494 308506624 6020 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411782 0 0 16781312 514 18224044590573256968 0 0 17 7 0 0 0
/proc/12828/statm: 75319 6020 372 364 0 11231 0
[pid=12830] ppid=12828 vsize=420220 CPUtime=33.78 cores=1,3,5,7
/proc/12830/stat : 12830 (adaptg2wsat2009) R 12828 12830 12830 0 -1 4202496 1669 0 0 0 3378 0 0 0 25 0 1 0 616961495 430305280 1400 33554432000 134512640 135107411 4290924544 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12830/statm: 105055 1400 59 146 0 103931 0
[pid=12834] ppid=12809 vsize=312140 CPUtime=0.03 cores=1,3,5,7
/proc/12834/stat : 12834 (python) S 12809 12809 12773 0 -1 4202560 932 0 0 0 2 1 0 0 18 0 1 0 616961497 319631360 6024 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411782 0 0 16781312 514 18224044590573256968 0 0 17 5 0 0 0
/proc/12834/statm: 78035 6024 372 364 0 13944 0
[pid=12836] ppid=12834 vsize=420220 CPUtime=33.67 cores=1,3,5,7
/proc/12836/stat : 12836 (adaptg2wsat2009) R 12834 12836 12836 0 -1 4202496 1665 0 0 0 3367 0 0 0 25 0 1 0 616961497 430305280 1400 33554432000 134512640 135107411 4288013792 18446744073709551615 134532302 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12836/statm: 105055 1400 59 146 0 103931 0
[pid=12809/tid=12823] ppid=12807 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12823/stat : 12823 (python) S 12807 12809 12773 0 -1 4202560 25 928 0 8 0 0 0 0 19 0 5 0 616961488 329003008 7538 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
[pid=12809/tid=12826] ppid=12807 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12826/stat : 12826 (python) S 12807 12809 12773 0 -1 4202560 21 928 0 8 0 0 0 0 21 0 5 0 616961491 329003008 7538 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
[pid=12809/tid=12829] ppid=12807 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12829/stat : 12829 (python) S 12807 12809 12773 0 -1 4202560 21 928 0 8 0 0 0 0 20 0 5 0 616961494 329003008 7538 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
[pid=12809/tid=12835] ppid=12807 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12835/stat : 12835 (python) S 12807 12809 12773 0 -1 4202560 27 928 0 8 0 0 0 0 22 0 5 0 616961497 329003008 7538 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 137.23
Current children cumulated vsize (KiB) 4386416

[startup+102.308 s]
/proc/loadavg: 6.29 2.09 0.74 10/202 12847
/proc/meminfo: memFree=29292736/32950928 swapFree=67111364/67111528
[pid=12809] ppid=12807 vsize=321292 CPUtime=2.3 cores=1,3,5,7
/proc/12809/stat : 12809 (python) S 12807 12809 12773 0 -1 4202496 15282 928 114 8 196 34 0 0 22 0 5 0 616959763 329003008 7538 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 361
/proc/12809/statm: 80323 7538 1883 364 0 16232 0
[pid=12822] ppid=12809 vsize=280764 CPUtime=0.1 cores=1,3,5,7
/proc/12822/stat : 12822 (python) S 12809 12809 12773 0 -1 4202560 911 0 0 0 5 5 0 0 18 0 1 0 616961488 287502336 6001 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411727 0 0 16781312 514 18224044590573256968 0 0 17 7 0 0 0
/proc/12822/statm: 70191 6001 378 364 0 6109 0
[pid=12824] ppid=12822 vsize=1019708 CPUtime=84.28 cores=1,3,5,7
/proc/12824/stat : 12824 (gnovelty+2) R 12822 12824 12824 0 -1 4202496 954 0 0 0 8428 0 0 0 25 0 1 0 616961489 1044180992 689 33554432000 134512640 135663704 4293165872 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12824/statm: 254927 689 113 282 0 254643 0
[pid=12825] ppid=12809 vsize=291088 CPUtime=0.11 cores=1,3,5,7
/proc/12825/stat : 12825 (python) S 12809 12809 12773 0 -1 4202560 922 0 0 0 6 5 0 0 18 0 1 0 616961491 298074112 5999 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411782 0 0 16781312 514 18224044590573256968 0 0 17 5 0 0 0
/proc/12825/statm: 72772 5999 372 364 0 8687 0

################
# More data... #
################

/proc/12828/task/12861/stat : 12861 (python) S 12809 12809 12773 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 616971507 308506624 6049 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604471127571136 0 0 -1 5 0 0 0
[pid=12834] ppid=12809 vsize=312140 CPUtime=0.13 cores=1,3,5,7
/proc/12834/stat : 12834 (python) S 12809 12809 12773 0 -1 4202560 1109 0 0 0 7 6 0 0 19 0 2 0 616961497 319631360 6053 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/12834/statm: 78035 6053 400 364 0 13944 0
[pid=12836] ppid=12834 vsize=420220 CPUtime=98.89 cores=1,3,5,7
/proc/12836/stat : 12836 (adaptg2wsat2009) T 12834 12836 12836 0 -1 4202496 1665 0 0 0 9889 0 0 0 25 0 1 0 616961497 430305280 1400 33554432000 134512640 135107411 4288013792 18446744073709551615 134532128 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/12836/statm: 105055 1400 59 146 0 103931 0
[pid=12834/tid=12854] ppid=12809 vsize=312140 CPUtime=0 cores=1,3,5,7
/proc/12834/task/12854/stat : 12854 (python) S 12809 12809 12773 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 616971413 319631360 6053 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604455026472896 0 0 -1 7 0 0 0
[pid=12851] ppid=12809 vsize=321940 CPUtime=0.13 cores=1,3,5,7
/proc/12851/stat : 12851 (python) S 12809 12809 12773 0 -1 4202560 1113 0 0 0 6 7 0 0 19 0 2 0 616971410 329666560 6057 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/12851/statm: 80485 6057 400 364 0 16391 0
[pid=12853] ppid=12851 vsize=420220 CPUtime=99.98 cores=1,3,5,7
/proc/12853/stat : 12853 (adaptg2wsat2009) T 12851 12853 12853 0 -1 4202496 1668 0 0 0 9998 0 0 0 25 0 1 0 616971410 430305280 1399 33554432000 134512640 135107411 4289568144 18446744073709551615 134531582 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12853/statm: 105055 1399 59 146 0 103931 0
[pid=12851/tid=12874] ppid=12809 vsize=321940 CPUtime=0 cores=1,3,5,7
/proc/12851/task/12874/stat : 12874 (python) S 12809 12809 12773 0 -1 4202560 52 0 0 0 0 0 0 0 19 0 2 0 616981422 329666560 6057 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604471160546432 0 0 -1 3 0 0 0
[pid=12855] ppid=12809 vsize=332360 CPUtime=0.13 cores=1,3,5,7
/proc/12855/stat : 12855 (python) S 12809 12809 12773 0 -1 4202560 1132 0 0 0 6 7 0 0 19 0 2 0 616971414 340336640 6082 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/12855/statm: 83090 6082 400 364 0 18993 0
[pid=12857] ppid=12855 vsize=1019708 CPUtime=98.42 cores=1,3,5,7
/proc/12857/stat : 12857 (gnovelty+2) T 12855 12857 12857 0 -1 4202496 954 0 0 0 9841 1 0 0 25 0 1 0 616971414 1044180992 689 33554432000 134512640 135663704 4291952288 18446744073709551615 134525276 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/12857/statm: 254927 689 113 282 0 254643 0
[pid=12855/tid=12879] ppid=12809 vsize=332360 CPUtime=0 cores=1,3,5,7
/proc/12855/task/12879/stat : 12879 (python) S 12809 12809 12773 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 616981431 340336640 6082 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604471160545536 0 0 -1 5 0 0 0
[pid=12858] ppid=12809 vsize=342456 CPUtime=0.13 cores=1,3,5,7
/proc/12858/stat : 12858 (python) S 12809 12809 12773 0 -1 4202560 1174 0 0 0 7 6 0 0 19 0 2 0 616971418 350674944 6133 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/12858/statm: 85614 6133 406 364 0 21514 0
[pid=12860] ppid=12858 vsize=249868 CPUtime=100 cores=1,3,5,7
/proc/12860/stat : 12860 (FH) T 12858 12860 12860 0 -1 4202496 1530 0 0 0 10000 0 0 0 25 0 1 0 616971418 255864832 1263 33554432000 134512640 134936988 4287774064 18446744073709551615 134545350 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/12860/statm: 62467 1263 41 104 0 61971 0
[pid=12858/tid=12882] ppid=12809 vsize=342456 CPUtime=0 cores=1,3,5,7
/proc/12858/task/12882/stat : 12882 (python) S 12809 12809 12773 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 616981433 350674944 6133 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604466158328768 0 0 -1 7 0 0 0
[pid=12862] ppid=12809 vsize=353320 CPUtime=0.13 cores=1,3,5,7
/proc/12862/stat : 12862 (python) S 12809 12809 12773 0 -1 4202560 1166 0 0 0 5 8 0 0 19 0 2 0 616971511 361799680 6137 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/12862/statm: 88330 6137 406 364 0 24227 0
[pid=12864] ppid=12862 vsize=420220 CPUtime=99.14 cores=1,3,5,7
/proc/12864/stat : 12864 (adaptg2wsat2009) T 12862 12864 12864 0 -1 4202496 1671 0 0 0 9914 0 0 0 25 0 1 0 616971511 430305280 1399 33554432000 134512640 135107411 4292521600 18446744073709551615 134532376 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/12864/statm: 105055 1399 59 146 0 103931 0
[pid=12862/tid=12875] ppid=12809 vsize=353320 CPUtime=0 cores=1,3,5,7
/proc/12862/task/12875/stat : 12875 (python) S 12809 12809 12773 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 616981426 361799680 6137 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604466158327872 0 0 -1 1 0 0 0
[pid=12876] ppid=12809 vsize=363092 CPUtime=0.14 cores=1,3,5,7
/proc/12876/stat : 12876 (python) S 12809 12809 12773 0 -1 4202560 1164 0 0 0 7 7 0 0 19 0 2 0 616981428 371806208 6246 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/12876/statm: 90773 6246 399 364 0 26667 0
[pid=12878] ppid=12876 vsize=420220 CPUtime=99.98 cores=1,3,5,7
/proc/12878/stat : 12878 (adaptg2wsat2009) T 12876 12878 12878 0 -1 4202496 1675 0 0 0 9998 0 0 0 25 0 1 0 616981428 430305280 1399 33554432000 134512640 135107411 4291115904 18446744073709551615 134531808 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12878/statm: 105055 1399 59 146 0 103931 0
[pid=12876/tid=12903] ppid=12809 vsize=363092 CPUtime=0 cores=1,3,5,7
/proc/12876/task/12903/stat : 12903 (python) S 12809 12809 12773 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 616991442 371806208 6246 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604469896967296 0 0 -1 3 0 0 0
[pid=12880] ppid=12809 vsize=373652 CPUtime=0.15 cores=1,3,5,7
/proc/12880/stat : 12880 (python) S 12809 12809 12773 0 -1 4202560 1162 0 0 0 8 7 0 0 19 0 2 0 616981432 382619648 6251 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/12880/statm: 93413 6251 400 364 0 29304 0
[pid=12883] ppid=12880 vsize=249876 CPUtime=98.36 cores=1,3,5,7
/proc/12883/stat : 12883 (FH) T 12880 12883 12883 0 -1 4202496 1537 0 0 0 9836 0 0 0 25 0 1 0 616981433 255873024 1264 33554432000 134512640 134936988 4294936288 18446744073709551615 134552486 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/12883/statm: 62469 1264 41 104 0 61973 0
[pid=12880/tid=12905] ppid=12809 vsize=373652 CPUtime=0 cores=1,3,5,7
/proc/12880/task/12905/stat : 12905 (python) S 12809 12809 12773 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 616991446 382619648 6251 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604465607715584 0 0 -1 5 0 0 0
[pid=12884] ppid=12809 vsize=383604 CPUtime=0.43 cores=1,3,5,7
/proc/12884/stat : 12884 (python) S 12809 12809 12773 0 -1 4202560 1183 0 0 0 22 21 0 0 18 0 2 0 616981436 392810496 6254 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/12884/statm: 95901 6254 399 364 0 31789 0
[pid=12886] ppid=12884 vsize=268420 CPUtime=291.14 cores=1,3,5,7
/proc/12886/stat : 12886 (hybridGM3) T 12884 12886 12886 0 -1 4202496 5393 0 0 0 29113 1 0 0 25 0 1 0 616981437 274862080 5125 33554432000 134512640 135204385 4287340368 18446744073709551615 134538000 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/12886/statm: 67105 5125 63 169 0 66935 0
[pid=12884/tid=12962] ppid=12809 vsize=383604 CPUtime=0 cores=1,3,5,7
/proc/12884/task/12962/stat : 12962 (python) S 12809 12809 12773 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 617001070 392810496 6254 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604462006393856 0 0 -1 5 0 0 0
[pid=12888] ppid=12809 vsize=394144 CPUtime=0.14 cores=1,3,5,7
/proc/12888/stat : 12888 (python) S 12809 12809 12773 0 -1 4202560 1160 0 0 0 8 6 0 0 19 0 2 0 616981441 403603456 6259 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/12888/statm: 98536 6259 400 364 0 34421 0
[pid=12890] ppid=12888 vsize=1019708 CPUtime=100.18 cores=1,3,5,7
/proc/12890/stat : 12890 (gnovelty+2) T 12888 12890 12890 0 -1 4202496 962 0 0 0 10018 0 0 0 25 0 1 0 616981441 1044180992 689 33554432000 134512640 135663704 4288164416 18446744073709551615 134525243 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/12890/statm: 254927 689 113 282 0 254643 0
[pid=12888/tid=12911] ppid=12809 vsize=394144 CPUtime=0 cores=1,3,5,7
/proc/12888/task/12911/stat : 12911 (python) S 12809 12809 12773 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 616991459 403603456 6259 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604455026473792 0 0 -1 7 0 0 0
[pid=12904] ppid=12809 vsize=404532 CPUtime=0.29 cores=1,3,5,7
/proc/12904/stat : 12904 (python) S 12809 12809 12773 0 -1 4202560 1026 0 0 0 14 15 0 0 18 0 1 0 616991446 414240768 6234 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411782 0 0 16781312 514 18224044590573256967 0 0 17 7 0 0 0
/proc/12904/statm: 101133 6234 372 364 0 37015 0
[pid=12907] ppid=12904 vsize=268428 CPUtime=192.34 cores=1,3,5,7
/proc/12907/stat : 12907 (hybridGM3) R 12904 12907 12907 0 -1 4202496 5396 0 0 0 19234 0 0 0 25 0 1 0 616991447 274870272 5127 33554432000 134512640 135204385 4293056992 18446744073709551615 134538375 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12907/statm: 67107 5127 63 169 0 66937 0
[pid=12908] ppid=12809 vsize=414544 CPUtime=0.14 cores=1,3,5,7
/proc/12908/stat : 12908 (python) S 12809 12809 12773 0 -1 4202560 1168 0 0 0 7 7 0 0 19 0 2 0 616991451 424493056 6267 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/12908/statm: 103636 6267 400 364 0 39515 0
[pid=12910] ppid=12908 vsize=420220 CPUtime=99.83 cores=1,3,5,7
/proc/12910/stat : 12910 (adaptg2wsat2009) T 12908 12910 12910 0 -1 4202496 1676 0 0 0 9983 0 0 0 25 0 1 0 616991451 430305280 1400 33554432000 134512640 135107411 4288315408 18446744073709551615 134531513 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/12910/statm: 105055 1400 59 146 0 103931 0
[pid=12908/tid=12963] ppid=12809 vsize=414544 CPUtime=0 cores=1,3,5,7
/proc/12908/task/12963/stat : 12963 (python) S 12809 12809 12773 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 617001465 424493056 6267 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604471147313344 0 0 -1 1 0 0 0
[pid=12912] ppid=12809 vsize=424948 CPUtime=0.29 cores=1,3,5,7
/proc/12912/stat : 12912 (python) S 12809 12809 12773 0 -1 4202560 1226 0 0 0 14 15 0 0 19 0 2 0 616991464 435146752 6279 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/12912/statm: 106237 6279 408 364 0 42113 0
[pid=12914] ppid=12912 vsize=268428 CPUtime=196.04 cores=1,3,5,7
/proc/12914/stat : 12914 (hybridGM3) T 12912 12914 12914 0 -1 4202496 5403 0 0 0 19604 0 0 0 25 0 1 0 616991464 274870272 5127 33554432000 134512640 135204385 4294481552 18446744073709551615 134538375 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/12914/statm: 67107 5127 63 169 0 66937 0
[pid=12912/tid=12976] ppid=12809 vsize=424948 CPUtime=0 cores=1,3,5,7
/proc/12912/task/12976/stat : 12976 (python) S 12809 12809 12773 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 617011097 435146752 6279 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 514 18446604469896968192 0 0 -1 5 0 0 0
[pid=12964] ppid=12809 vsize=435344 CPUtime=0.13 cores=1,3,5,7
/proc/12964/stat : 12964 (python) S 12809 12809 12773 0 -1 4202560 1066 0 0 0 7 6 0 0 18 0 1 0 617001470 445792256 6292 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411782 0 0 16781312 514 18224044590573256967 0 0 17 3 0 0 0
/proc/12964/statm: 108836 6292 380 364 0 44709 0
[pid=12966] ppid=12964 vsize=268428 CPUtime=96.26 cores=1,3,5,7
/proc/12966/stat : 12966 (hybridGM3) R 12964 12966 12966 0 -1 4202496 5404 0 0 0 9626 0 0 0 25 0 1 0 617001470 274870272 5127 33554432000 134512640 135204385 4287633984 18446744073709551615 134538248 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12966/statm: 67107 5127 63 169 0 66937 0
[pid=12973] ppid=12809 vsize=445764 CPUtime=0 cores=1,3,5,7
/proc/12973/stat : 12973 (python) S 12809 12809 12773 0 -1 4202560 1011 0 0 0 0 0 0 0 18 0 1 0 617010592 456462336 6290 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411782 0 0 16781312 514 4502148826072580360 0 0 17 3 0 0 0
/proc/12973/statm: 111441 6290 374 364 0 47311 0
[pid=12975] ppid=12973 vsize=458348 CPUtime=5.29 cores=1,3,5,7
/proc/12975/stat : 12975 (TNM) R 12973 12975 12975 0 -1 4202496 1515 0 0 0 529 0 0 0 25 0 1 0 617010592 469348352 1240 33554432000 134512640 135008748 4289852336 18446744073709551615 134527734 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/12975/statm: 114587 1240 46 122 0 113487 0
[pid=12977] ppid=12809 vsize=456112 CPUtime=0 cores=1,3,5,7
/proc/12977/stat : 12977 (python) S 12809 12809 12773 0 -1 4202560 825 0 0 0 0 0 0 0 19 0 1 0 617011102 467058688 6267 33554432000 4194304 5683644 140733432923968 18446744073709551615 256415411782 0 0 16781312 514 5628070448269885439 0 0 17 3 0 0 0
/proc/12977/statm: 114028 6267 347 364 0 49895 0
[pid=12979] ppid=12977 vsize=247956 CPUtime=0.22 cores=1,3,5,7
/proc/12979/stat : 12979 (FH) R 12977 12979 12979 0 -1 4202496 1259 0 0 0 22 0 0 0 21 0 1 0 617011103 253906944 986 33554432000 134512640 134936988 4292347152 18446744073709551615 134523622 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12979/statm: 61989 986 37 104 0 61493 0
[pid=12809/tid=12823] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12823/stat : 12823 (python) S 12807 12809 12773 0 -1 4202560 25 928 0 8 0 0 0 0 19 0 19 0 616961488 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
[pid=12809/tid=12826] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12826/stat : 12826 (python) S 12807 12809 12773 0 -1 4202560 21 928 0 8 0 0 0 0 21 0 19 0 616961491 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
[pid=12809/tid=12829] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12829/stat : 12829 (python) S 12807 12809 12773 0 -1 4202560 21 928 0 8 0 0 0 0 20 0 19 0 616961494 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
[pid=12809/tid=12835] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12835/stat : 12835 (python) S 12807 12809 12773 0 -1 4202560 27 928 0 8 0 0 0 0 22 0 19 0 616961497 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
[pid=12809/tid=12852] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12852/stat : 12852 (python) S 12807 12809 12773 0 -1 4202560 25 928 0 8 0 0 0 0 20 0 19 0 616971410 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 3 0 0 0
[pid=12809/tid=12856] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12856/stat : 12856 (python) S 12807 12809 12773 0 -1 4202560 29 928 0 8 0 0 0 0 20 0 19 0 616971414 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 7 0 0 0
[pid=12809/tid=12859] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12859/stat : 12859 (python) S 12807 12809 12773 0 -1 4202560 30 928 0 8 0 0 0 0 22 0 19 0 616971418 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 7 0 0 0
[pid=12809/tid=12863] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12863/stat : 12863 (python) S 12807 12809 12773 0 -1 4202560 31 928 0 8 0 0 0 0 20 0 19 0 616971511 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 7 0 0 0
[pid=12809/tid=12877] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12877/stat : 12877 (python) S 12807 12809 12773 0 -1 4202560 23 928 0 8 0 0 0 0 20 0 19 0 616981428 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 3 0 0 0
[pid=12809/tid=12881] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12881/stat : 12881 (python) S 12807 12809 12773 0 -1 4202560 23 928 0 8 0 0 0 0 22 0 19 0 616981432 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 1 0 0 0
[pid=12809/tid=12885] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12885/stat : 12885 (python) S 12807 12809 12773 0 -1 4202560 29 928 0 8 0 0 0 0 18 0 19 0 616981436 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 5 0 0 0
[pid=12809/tid=12889] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12889/stat : 12889 (python) S 12807 12809 12773 0 -1 4202560 29 928 0 8 0 0 0 0 20 0 19 0 616981441 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 7 0 0 0
[pid=12809/tid=12906] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12906/stat : 12906 (python) S 12807 12809 12773 0 -1 4202560 24 928 0 8 0 0 0 0 20 0 19 0 616991447 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 7 0 0 0
[pid=12809/tid=12909] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12909/stat : 12909 (python) S 12807 12809 12773 0 -1 4202560 30 928 0 8 0 0 0 0 20 0 19 0 616991451 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 3 0 0 0
[pid=12809/tid=12913] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12913/stat : 12913 (python) S 12807 12809 12773 0 -1 4202560 27 928 0 8 0 0 0 0 21 0 19 0 616991464 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 3 0 0 0
[pid=12809/tid=12965] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12965/stat : 12965 (python) S 12807 12809 12773 0 -1 4202560 29 928 0 8 0 0 0 0 20 0 19 0 617001470 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 3 0 0 0
[pid=12809/tid=12974] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12974/stat : 12974 (python) S 12807 12809 12773 0 -1 4202560 25 928 0 8 0 0 0 0 20 0 19 0 617010592 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 3 0 0 0
[pid=12809/tid=12978] ppid=12807 vsize=465792 CPUtime=0 cores=1,3,5,7
/proc/12809/task/12978/stat : 12978 (python) S 12807 12809 12773 0 -1 4202560 25 928 0 8 0 0 0 0 20 0 19 0 617011103 476971008 7835 33554432000 4194304 5683644 140733432923968 18446744073709551615 256427215281 0 0 16781312 2 18446604471132419136 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1979.32
Current children cumulated vsize (KiB) 15987020

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 12809 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=593605
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=389940
# CPU time returned by wait4() is 2.98354
# while last known CPU time is 1979.32
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 513.73
CPU time (s): 1979.32
CPU user time (s): 1977.55
CPU system time (s): 1.77
CPU usage (%): 385.284
Max. virtual memory (cumulated for all children) (KiB): 15987020

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.59361
system time used= 0.38994
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 30197
page faults= 122
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 17865
involuntary context switches= 129

runsolver used 2.30165 second user time and 5.69013 second system time

The end

Launcher Data 

Begin job on node115 at 2011-05-01 18:28:06
IDJOB=3317081
IDBENCH=82546
IDSOLVER=1705
FILE ID=node115/3317081-1304267286
RUNJOBID= node115-1304267285-12792
PBS_JOBID= 13173964
Free space on /tmp= 73164 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k7-r85-v300-c25500-S1286362348-008-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317081-1304267286/watcher-3317081-1304267286 -o /tmp/evaluation-result-3317081-1304267286/solver-3317081-1304267286 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317081-1304267286.cnf -seed 1878399543 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 63b66954174098a654eacd27832fcf38
RANDOM SEED=1878399543

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      29517664 kB
Buffers:        872748 kB
Cached:        2063860 kB
SwapCached:          0 kB
Active:         410028 kB
Inactive:      2590048 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      29517664 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:           65244 kB
Writeback:           0 kB
AnonPages:       63820 kB
Mapped:          14552 kB
Slab:           368148 kB
PageTables:       4652 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   187216 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73068 MiB
End job on node115 at 2011-05-01 18:36:47