Trace number 3317110

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 1984.32 500.519

General information on the benchmark

Namerandom/large/
unif-k5-r20-v2000-c40000-S394483079-002-UNKNOWN.cnf
MD5SUM40e955d306a70bd3dffa8283c1221639
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2000
Number of clauses40000
Sum of the clauses size200000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 540000
Number of clauses of size over 50

Solver Data 

0.40/0.66	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.40/0.66	c INFO: solving HOME/instance-3317110-1304267328.cnf
0.40/0.66	c INFO: computing task features
0.58/0.89	c INFO: computed variable-clause graph statistics
0.58/0.89	c INFO: computed clause balance statistics
1.77/2.02	c INFO: computed variable balance statistics
2.37/2.65	c INFO: computed small-clause counts
2.37/2.65	c INFO: computed Horn-clause counts
2.47/2.70	c INFO: computed variable graph statistics
2.47/2.70	c INFO: collected features for HOME/instance-3317110-1304267328.cnf in 2.03 s
2.47/2.73	c INFO: running gnovelty+2@0 for 97 with 4797 remaining (b = 0.38)
2.47/2.76	c INFO: running gnovelty+2@0 for 97 with 4797 remaining (b = 0.13)
2.47/2.79	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.08)
2.57/2.82	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.07)
402.42/102.93	c INFO: running adaptg2wsat2009++@0 for 97 with 4697 remaining (b = 0.05)
402.42/102.98	c INFO: running gnovelty+2@0 for 97 with 4697 remaining (b = 0.05)
402.64/103.01	c INFO: running FH@0 for 97 with 4697 remaining (b = 0.03)
402.64/103.05	c INFO: running adaptg2wsat2009++@0 for 97 with 4697 remaining (b = 0.03)
797.39/202.11	c INFO: running adaptg2wsat2009++@0 for 97 with 4598 remaining (b = 0.02)
801.36/203.18	c INFO: running FH@0 for 97 with 4597 remaining (b = 0.02)
801.71/203.22	c INFO: running hybridGM3@0 for 194 with 4597 remaining (b = 0.04)
801.71/203.27	c INFO: running gnovelty+2@0 for 97 with 4597 remaining (b = 0.02)
1195.96/302.30	c INFO: running hybridGM3@0 for 194 with 4498 remaining (b = 0.03)
1200.33/303.37	c INFO: running adaptg2wsat2009++@0 for 97 with 4497 remaining (b = 0.01)
1201.13/303.51	c INFO: running hybridGM3@0 for 194 with 4497 remaining (b = 0.03)
1580.18/398.71	c INFO: running hybridGM3@99 for 97 with 4401 remaining (b = 0.01)
1599.30/403.56	c INFO: running hybridGM3@0 for 194 with 4397 remaining (b = 0.02)
1973.57/497.74	c INFO: running hybridGM3@100 for 97 with 4302 remaining (b = 0.01)
1982.31/499.90	c INFO: running TNM@0 for 1552 with 4300 remaining (b = 0.19)
1982.31/499.97	c INFO: running FH@0 for 97 with 4300 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317110-1304267328/watcher-3317110-1304267328 -o /tmp/evaluation-result-3317110-1304267328/solver-3317110-1304267328 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317110-1304267328.cnf -seed 2121985107 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.60 0.63 0.21 6/187 28737
/proc/meminfo: memFree=30307372/32951132 swapFree=67080200/67111528
[pid=28737] ppid=28735 vsize=15184 CPUtime=0 cores=0,2,4,6
/proc/28737/stat : 28737 (python) S 28735 28737 28639 0 -1 4202496 723 0 0 0 0 0 0 0 25 0 1 0 538332492 15548416 614 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647144448 0 2147483391 16781312 2 18446744071699109054 0 0 17 2 0 0 0
/proc/28737/statm: 3796 614 321 364 0 283 0

[startup+0.0250509 s]
/proc/loadavg: 2.60 0.63 0.21 6/187 28737
/proc/meminfo: memFree=30307372/32951132 swapFree=67080200/67111528
[pid=28737] ppid=28735 vsize=16392 CPUtime=0.01 cores=0,2,4,6
/proc/28737/stat : 28737 (python) S 28735 28737 28639 0 -1 4202496 1078 0 0 0 1 0 0 0 25 0 1 0 538332492 16785408 949 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647144448 0 2147483391 16781312 2 18446744071699109054 0 0 17 2 0 0 0
/proc/28737/statm: 4098 949 385 364 0 585 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16392

[startup+0.101019 s]
/proc/loadavg: 2.60 0.63 0.21 6/187 28737
/proc/meminfo: memFree=30307372/32951132 swapFree=67080200/67111528
[pid=28737] ppid=28735 vsize=89764 CPUtime=0.06 cores=0,2,4,6
/proc/28737/stat : 28737 (python) R 28735 28737 28639 0 -1 4202496 2671 0 0 0 4 2 0 0 17 0 1 0 538332492 91918336 2387 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647144448 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/28737/statm: 22441 2387 750 364 0 1621 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 89764

[startup+0.300982 s]
/proc/loadavg: 2.60 0.63 0.21 6/187 28737
/proc/meminfo: memFree=30307372/32951132 swapFree=67080200/67111528
[pid=28737] ppid=28735 vsize=135748 CPUtime=0.21 cores=0,2,4,6
/proc/28737/stat : 28737 (python) S 28735 28737 28639 0 -1 4202496 5964 935 1 0 14 7 0 0 17 0 1 0 538332492 139005952 4151 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647142405 0 2147483391 16781312 2 18446744071699109054 0 0 17 0 0 0 0
/proc/28737/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 135748

[startup+0.70089 s]
/proc/loadavg: 2.60 0.63 0.21 6/187 28737
/proc/meminfo: memFree=30307372/32951132 swapFree=67080200/67111528
[pid=28737] ppid=28735 vsize=280800 CPUtime=0.48 cores=0,2,4,6
/proc/28737/stat : 28737 (python) R 28735 28737 28639 0 -1 4202496 9870 935 1 0 34 14 0 0 16 0 1 0 538332492 287539200 7549 33554432000 4194304 5683644 140737046920928 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/28737/statm: 70200 7549 1761 364 0 6124 0
Current children cumulated CPU time (s) 0.48
Current children cumulated vsize (KiB) 280800

[startup+1.50075 s]
/proc/loadavg: 2.60 0.63 0.21 9/188 28743
/proc/meminfo: memFree=30271672/32951132 swapFree=67080200/67111528
[pid=28737] ppid=28735 vsize=290796 CPUtime=1.27 cores=0,2,4,6
/proc/28737/stat : 28737 (python) R 28735 28737 28639 0 -1 4202496 14011 935 1 0 113 14 0 0 16 0 1 0 538332492 297775104 10084 33554432000 4194304 5683644 140737046920928 18446744073709551615 4415616 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/28737/statm: 72699 10084 1826 364 0 8623 0
Current children cumulated CPU time (s) 1.27
Current children cumulated vsize (KiB) 290796

[startup+3.10045 s]
/proc/loadavg: 2.95 0.74 0.25 7/188 28743
/proc/meminfo: memFree=30271820/32951132 swapFree=67080200/67111528
[pid=28737] ppid=28735 vsize=325508 CPUtime=2.6 cores=0,2,4,6
/proc/28737/stat : 28737 (python) S 28735 28737 28639 0 -1 4202496 16927 935 1 0 245 15 0 0 24 0 5 0 538332492 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28737/statm: 81377 8576 1883 364 0 17286 0
Current children cumulated CPU time (s) 2.6
Current children cumulated vsize (KiB) 325508

[startup+6.30091 s]
/proc/loadavg: 2.95 0.74 0.25 11/200 28755
/proc/meminfo: memFree=30238112/32951132 swapFree=67080200/67111528
[pid=28737] ppid=28735 vsize=325508 CPUtime=2.6 cores=0,2,4,6
/proc/28737/stat : 28737 (python) S 28735 28737 28639 0 -1 4202496 16927 935 1 0 245 15 0 0 24 0 5 0 538332492 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28737/statm: 81377 8576 1883 364 0 17286 0
[pid=28744] ppid=28737 vsize=284980 CPUtime=0 cores=0,2,4,6
/proc/28744/stat : 28744 (python) S 28737 28737 28639 0 -1 4202560 726 0 0 0 0 0 0 0 18 0 1 0 538332765 291819520 6898 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166479 0 0 16781312 514 7404692947389612289 0 0 17 0 0 0 0
/proc/28744/statm: 71245 6898 355 364 0 7163 0
[pid=28746] ppid=28744 vsize=1020308 CPUtime=3.56 cores=0,2,4,6
/proc/28746/stat : 28746 (gnovelty+2) R 28744 28746 28746 0 -1 4202496 1157 0 0 0 356 0 0 0 25 0 1 0 538332765 1044795392 892 33554432000 134512640 135663704 4292732752 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/28746/statm: 255077 892 113 282 0 254793 0
[pid=28747] ppid=28737 vsize=295000 CPUtime=0 cores=0,2,4,6
/proc/28747/stat : 28747 (python) S 28737 28737 28639 0 -1 4202560 745 0 0 0 0 0 0 0 18 0 1 0 538332768 302080000 7031 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 12581008423249740035 0 0 17 4 0 0 0
/proc/28747/statm: 73750 7031 349 364 0 9665 0
[pid=28749] ppid=28747 vsize=1020308 CPUtime=3.51 cores=0,2,4,6
/proc/28749/stat : 28749 (gnovelty+2) R 28747 28749 28749 0 -1 4202496 1153 0 0 0 351 0 0 0 25 0 1 0 538332768 1044795392 891 33554432000 134512640 135663704 4288418736 18446744073709551615 134545644 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/28749/statm: 255077 891 113 282 0 254793 0
[pid=28750] ppid=28737 vsize=305008 CPUtime=0 cores=0,2,4,6
/proc/28750/stat : 28750 (python) S 28737 28737 28639 0 -1 4202560 741 0 0 0 0 0 0 0 18 0 1 0 538332771 312328192 7033 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 4330958713918554371 0 0 17 0 0 0 0
/proc/28750/statm: 76252 7033 347 364 0 12164 0
[pid=28752] ppid=28750 vsize=423400 CPUtime=3.49 cores=0,2,4,6
/proc/28752/stat : 28752 (adaptg2wsat2009) R 28750 28752 28752 0 -1 4202496 2623 0 0 0 349 0 0 0 25 0 1 0 538332771 433561600 2357 33554432000 134512640 135107411 4290514304 18446744073709551615 134531613 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/28752/statm: 105850 2357 59 146 0 104726 0
[pid=28753] ppid=28737 vsize=315984 CPUtime=0 cores=0,2,4,6
/proc/28753/stat : 28753 (python) S 28737 28737 28639 0 -1 4202560 749 0 0 0 0 0 0 0 18 0 1 0 538332774 323567616 7039 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 3815974955258970371 0 0 17 4 0 0 0
/proc/28753/statm: 78996 7039 349 364 0 14905 0
[pid=28755] ppid=28753 vsize=423400 CPUtime=3.47 cores=0,2,4,6
/proc/28755/stat : 28755 (adaptg2wsat2009) R 28753 28755 28755 0 -1 4202496 2622 0 0 0 347 0 0 0 25 0 1 0 538332774 433561600 2358 33554432000 134512640 135107411 4289369040 18446744073709551615 134531848 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/28755/statm: 105850 2358 59 146 0 104726 0
[pid=28737/tid=28745] ppid=28735 vsize=325508 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28745/stat : 28745 (python) S 28735 28737 28639 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 5 0 538332765 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 0 0 0 0
[pid=28737/tid=28748] ppid=28735 vsize=325508 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28748/stat : 28748 (python) S 28735 28737 28639 0 -1 4202560 22 935 0 0 0 0 0 0 20 0 5 0 538332768 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28751] ppid=28735 vsize=325508 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28751/stat : 28751 (python) S 28735 28737 28639 0 -1 4202560 22 935 0 0 0 0 0 0 22 0 5 0 538332771 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28754] ppid=28735 vsize=325508 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28754/stat : 28754 (python) S 28735 28737 28639 0 -1 4202560 23 935 0 0 0 0 0 0 24 0 5 0 538332774 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 16.63
Current children cumulated vsize (KiB) 4413896

[startup+12.7008 s]
/proc/loadavg: 4.02 1.04 0.35 9/200 28755
/proc/meminfo: memFree=30237348/32951132 swapFree=67080200/67111528
[pid=28737] ppid=28735 vsize=325508 CPUtime=2.6 cores=0,2,4,6
/proc/28737/stat : 28737 (python) S 28735 28737 28639 0 -1 4202496 16927 935 1 0 245 15 0 0 24 0 5 0 538332492 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28737/statm: 81377 8576 1883 364 0 17286 0
[pid=28744] ppid=28737 vsize=284980 CPUtime=0 cores=0,2,4,6
/proc/28744/stat : 28744 (python) S 28737 28737 28639 0 -1 4202560 882 0 0 0 0 0 0 0 18 0 1 0 538332765 291819520 6921 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166479 0 0 16781312 514 7404692947389612289 0 0 17 0 0 0 0
/proc/28744/statm: 71245 6921 378 364 0 7163 0
[pid=28746] ppid=28744 vsize=1020308 CPUtime=9.95 cores=0,2,4,6
/proc/28746/stat : 28746 (gnovelty+2) R 28744 28746 28746 0 -1 4202496 1157 0 0 0 995 0 0 0 25 0 1 0 538332765 1044795392 892 33554432000 134512640 135663704 4292732752 18446744073709551615 134525213 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/28746/statm: 255077 892 113 282 0 254793 0
[pid=28747] ppid=28737 vsize=295000 CPUtime=0 cores=0,2,4,6
/proc/28747/stat : 28747 (python) S 28737 28737 28639 0 -1 4202560 916 0 0 0 0 0 0 0 18 0 1 0 538332768 302080000 7054 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 12581008423249740035 0 0 17 4 0 0 0
/proc/28747/statm: 73750 7054 372 364 0 9665 0
[pid=28749] ppid=28747 vsize=1020308 CPUtime=9.86 cores=0,2,4,6
/proc/28749/stat : 28749 (gnovelty+2) R 28747 28749 28749 0 -1 4202496 1153 0 0 0 985 1 0 0 25 0 1 0 538332768 1044795392 891 33554432000 134512640 135663704 4288418736 18446744073709551615 134524610 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/28749/statm: 255077 891 113 282 0 254793 0
[pid=28750] ppid=28737 vsize=305008 CPUtime=0 cores=0,2,4,6
/proc/28750/stat : 28750 (python) S 28737 28737 28639 0 -1 4202560 893 0 0 0 0 0 0 0 18 0 1 0 538332771 312328192 7055 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 4330958713918554371 0 0 17 0 0 0 0
/proc/28750/statm: 76252 7055 369 364 0 12164 0
[pid=28752] ppid=28750 vsize=423400 CPUtime=9.88 cores=0,2,4,6
/proc/28752/stat : 28752 (adaptg2wsat2009) R 28750 28752 28752 0 -1 4202496 2623 0 0 0 987 1 0 0 25 0 1 0 538332771 433561600 2357 33554432000 134512640 135107411 4290514304 18446744073709551615 134531561 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/28752/statm: 105850 2357 59 146 0 104726 0
[pid=28753] ppid=28737 vsize=315984 CPUtime=0 cores=0,2,4,6
/proc/28753/stat : 28753 (python) S 28737 28737 28639 0 -1 4202560 932 0 0 0 0 0 0 0 18 0 1 0 538332774 323567616 7062 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 3815974955258970371 0 0 17 4 0 0 0
/proc/28753/statm: 78996 7062 372 364 0 14905 0
[pid=28755] ppid=28753 vsize=423400 CPUtime=9.85 cores=0,2,4,6
/proc/28755/stat : 28755 (adaptg2wsat2009) R 28753 28755 28755 0 -1 4202496 2622 0 0 0 985 0 0 0 25 0 1 0 538332774 433561600 2358 33554432000 134512640 135107411 4289369040 18446744073709551615 134531404 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/28755/statm: 105850 2358 59 146 0 104726 0
[pid=28737/tid=28745] ppid=28735 vsize=325508 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28745/stat : 28745 (python) S 28735 28737 28639 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 5 0 538332765 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 0 0 0 0
[pid=28737/tid=28748] ppid=28735 vsize=325508 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28748/stat : 28748 (python) S 28735 28737 28639 0 -1 4202560 22 935 0 0 0 0 0 0 20 0 5 0 538332768 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28751] ppid=28735 vsize=325508 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28751/stat : 28751 (python) S 28735 28737 28639 0 -1 4202560 22 935 0 0 0 0 0 0 22 0 5 0 538332771 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28754] ppid=28735 vsize=325508 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28754/stat : 28754 (python) S 28735 28737 28639 0 -1 4202560 23 935 0 0 0 0 0 0 24 0 5 0 538332774 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 42.14
Current children cumulated vsize (KiB) 4413896

[startup+25.5007 s]
/proc/loadavg: 4.63 1.27 0.43 9/200 28756
/proc/meminfo: memFree=30237380/32951132 swapFree=67080200/67111528
[pid=28737] ppid=28735 vsize=325508 CPUtime=2.6 cores=0,2,4,6
/proc/28737/stat : 28737 (python) S 28735 28737 28639 0 -1 4202496 16927 935 1 0 245 15 0 0 24 0 5 0 538332492 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/28737/statm: 81377 8576 1883 364 0 17286 0
[pid=28744] ppid=28737 vsize=284980 CPUtime=0.02 cores=0,2,4,6
/proc/28744/stat : 28744 (python) S 28737 28737 28639 0 -1 4202560 890 0 0 0 1 1 0 0 18 0 1 0 538332765 291819520 6921 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166479 0 0 16781312 514 808695200275726593 0 0 17 0 0 0 0
/proc/28744/statm: 71245 6921 378 364 0 7163 0
[pid=28746] ppid=28744 vsize=1020308 CPUtime=22.75 cores=0,2,4,6
/proc/28746/stat : 28746 (gnovelty+2) R 28744 28746 28746 0 -1 4202496 1157 0 0 0 2275 0 0 0 25 0 1 0 538332765 1044795392 892 33554432000 134512640 135663704 4292732752 18446744073709551615 134545631 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/28746/statm: 255077 892 113 282 0 254793 0
[pid=28747] ppid=28737 vsize=295000 CPUtime=0.02 cores=0,2,4,6
/proc/28747/stat : 28747 (python) S 28737 28737 28639 0 -1 4202560 925 0 0 0 1 1 0 0 18 0 1 0 538332768 302080000 7054 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 808695200275726595 0 0 17 4 0 0 0
/proc/28747/statm: 73750 7054 372 364 0 9665 0
[pid=28749] ppid=28747 vsize=1020308 CPUtime=22.58 cores=0,2,4,6
/proc/28749/stat : 28749 (gnovelty+2) R 28747 28749 28749 0 -1 4202496 1153 0 0 0 2257 1 0 0 25 0 1 0 538332768 1044795392 891 33554432000 134512640 135663704 4288418736 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/28749/statm: 255077 891 113 282 0 254793 0
[pid=28750] ppid=28737 vsize=305008 CPUtime=0.03 cores=0,2,4,6
/proc/28750/stat : 28750 (python) S 28737 28737 28639 0 -1 4202560 901 0 0 0 2 1 0 0 18 0 1 0 538332771 312328192 7056 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 808695200275726595 0 0 17 0 0 0 0
/proc/28750/statm: 76252 7056 370 364 0 12164 0
[pid=28752] ppid=28750 vsize=423400 CPUtime=22.63 cores=0,2,4,6
/proc/28752/stat : 28752 (adaptg2wsat2009) R 28750 28752 28752 0 -1 4202496 2623 0 0 0 2262 1 0 0 25 0 1 0 538332771 433561600 2357 33554432000 134512640 135107411 4290514304 18446744073709551615 134539270 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/28752/statm: 105850 2357 59 146 0 104726 0
[pid=28753] ppid=28737 vsize=315984 CPUtime=0.02 cores=0,2,4,6
/proc/28753/stat : 28753 (python) S 28737 28737 28639 0 -1 4202560 945 0 0 0 1 1 0 0 18 0 1 0 538332774 323567616 7062 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 808695200275726595 0 0 17 4 0 0 0
/proc/28753/statm: 78996 7062 372 364 0 14905 0
[pid=28755] ppid=28753 vsize=423400 CPUtime=22.6 cores=0,2,4,6
/proc/28755/stat : 28755 (adaptg2wsat2009) R 28753 28755 28755 0 -1 4202496 2622 0 0 0 2260 0 0 0 25 0 1 0 538332774 433561600 2358 33554432000 134512640 135107411 4289369040 18446744073709551615 134532128 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/28755/statm: 105850 2358 59 146 0 104726 0
[pid=28737/tid=28745] ppid=28735 vsize=325508 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28745/stat : 28745 (python) S 28735 28737 28639 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 5 0 538332765 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 0 0 0 0
[pid=28737/tid=28748] ppid=28735 vsize=325508 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28748/stat : 28748 (python) S 28735 28737 28639 0 -1 4202560 22 935 0 0 0 0 0 0 20 0 5 0 538332768 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28751] ppid=28735 vsize=325508 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28751/stat : 28751 (python) S 28735 28737 28639 0 -1 4202560 22 935 0 0 0 0 0 0 22 0 5 0 538332771 333320192 8576 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0

################
# More data... #
################

/proc/28753/stat : 28753 (python) S 28737 28737 28639 0 -1 4202560 1124 0 0 0 7 5 0 0 19 0 2 0 538332774 323567616 7091 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/28753/statm: 78996 7091 400 364 0 14905 0
[pid=28755] ppid=28753 vsize=423400 CPUtime=99.85 cores=0,2,4,6
/proc/28755/stat : 28755 (adaptg2wsat2009) T 28753 28755 28755 0 -1 4202496 2622 0 0 0 9985 0 0 0 25 0 1 0 538332774 433561600 2358 33554432000 134512640 135107411 4289369040 18446744073709551615 134532349 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/28755/statm: 105850 2358 59 146 0 104726 0
[pid=28753/tid=28769] ppid=28737 vsize=315984 CPUtime=0 cores=0,2,4,6
/proc/28753/task/28769/stat : 28769 (python) S 28737 28737 28639 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 538342788 323567616 7091 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604453801169152 0 0 -1 4 0 0 0
[pid=28764] ppid=28737 vsize=325756 CPUtime=0.13 cores=0,2,4,6
/proc/28764/stat : 28764 (python) S 28737 28737 28639 0 -1 4202560 1132 0 0 0 7 6 0 0 19 0 2 0 538342785 333574144 7153 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/28764/statm: 81439 7153 399 364 0 17345 0
[pid=28768] ppid=28764 vsize=423400 CPUtime=98.97 cores=0,2,4,6
/proc/28768/stat : 28768 (adaptg2wsat2009) T 28764 28768 28768 0 -1 4202496 2623 0 0 0 9897 0 0 0 25 0 1 0 538342787 433561600 2357 33554432000 134512640 135107411 4294547776 18446744073709551615 134531613 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/28768/statm: 105850 2357 59 146 0 104726 0
[pid=28764/tid=28787] ppid=28737 vsize=325756 CPUtime=0 cores=0,2,4,6
/proc/28764/task/28787/stat : 28787 (python) S 28737 28737 28639 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 538352700 333574144 7153 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604453800931456 0 0 -1 6 0 0 0
[pid=28770] ppid=28737 vsize=336012 CPUtime=0.13 cores=0,2,4,6
/proc/28770/stat : 28770 (python) S 28737 28737 28639 0 -1 4202560 1143 0 0 0 8 5 0 0 19 0 2 0 538342790 344076288 7157 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/28770/statm: 84003 7157 399 364 0 19906 0
[pid=28772] ppid=28770 vsize=1020308 CPUtime=99.98 cores=0,2,4,6
/proc/28772/stat : 28772 (gnovelty+2) T 28770 28772 28772 0 -1 4202496 1155 0 0 0 9994 4 0 0 25 0 1 0 538342790 1044795392 891 33554432000 134512640 135663704 4289303680 18446744073709551615 134524925 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/28772/statm: 255077 891 113 282 0 254793 0
[pid=28770/tid=28792] ppid=28737 vsize=336012 CPUtime=0 cores=0,2,4,6
/proc/28770/task/28792/stat : 28792 (python) S 28737 28737 28639 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 538352807 344076288 7157 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604445065472768 0 0 -1 2 0 0 0
[pid=28773] ppid=28737 vsize=346896 CPUtime=0.13 cores=0,2,4,6
/proc/28773/stat : 28773 (python) S 28737 28737 28639 0 -1 4202560 1199 0 0 0 7 6 0 0 19 0 2 0 538342793 355221504 7168 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/28773/statm: 86724 7168 406 364 0 22624 0
[pid=28775] ppid=28773 vsize=253032 CPUtime=98.44 cores=0,2,4,6
/proc/28775/stat : 28775 (FH) T 28773 28775 28775 0 -1 4202496 2499 0 0 0 9844 0 0 0 25 0 1 0 538342793 259104768 2233 33554432000 134512640 134936988 4287655888 18446744073709551615 134547335 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/28775/statm: 63258 2233 41 104 0 62762 0
[pid=28773/tid=28791] ppid=28737 vsize=346896 CPUtime=0 cores=0,2,4,6
/proc/28773/task/28791/stat : 28791 (python) S 28737 28737 28639 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538352806 355221504 7168 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604447488910272 0 0 -1 4 0 0 0
[pid=28776] ppid=28737 vsize=356820 CPUtime=0.13 cores=0,2,4,6
/proc/28776/stat : 28776 (python) S 28737 28737 28639 0 -1 4202560 1185 0 0 0 8 5 0 0 19 0 2 0 538342797 365383680 7208 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/28776/statm: 89205 7208 406 364 0 25102 0
[pid=28778] ppid=28776 vsize=423400 CPUtime=99.95 cores=0,2,4,6
/proc/28778/stat : 28778 (adaptg2wsat2009) T 28776 28778 28778 0 -1 4202496 2629 0 0 0 9995 0 0 0 25 0 1 0 538342797 433561600 2357 33554432000 134512640 135107411 4291402368 18446744073709551615 134532128 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/28778/statm: 105850 2357 59 146 0 104726 0
[pid=28776/tid=28796] ppid=28737 vsize=356820 CPUtime=0 cores=0,2,4,6
/proc/28776/task/28796/stat : 28796 (python) S 28737 28737 28639 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 538352812 365383680 7208 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604445065470976 0 0 -1 0 0 0 0
[pid=28788] ppid=28737 vsize=366924 CPUtime=0.15 cores=0,2,4,6
/proc/28788/stat : 28788 (python) S 28737 28737 28639 0 -1 4202560 1149 0 0 0 10 5 0 0 19 0 2 0 538352703 375730176 7205 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/28788/statm: 91731 7205 399 364 0 27625 0
[pid=28790] ppid=28788 vsize=423400 CPUtime=99.95 cores=0,2,4,6
/proc/28790/stat : 28790 (adaptg2wsat2009) T 28788 28790 28790 0 -1 4202496 2628 0 0 0 9995 0 0 0 25 0 1 0 538352704 433561600 2357 33554432000 134512640 135107411 4294519984 18446744073709551615 134531709 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/28790/statm: 105850 2357 59 146 0 104726 0
[pid=28788/tid=28814] ppid=28737 vsize=366924 CPUtime=0 cores=0,2,4,6
/proc/28788/task/28814/stat : 28814 (python) S 28737 28737 28639 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538362717 375730176 7205 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604453868602304 0 0 -1 0 0 0 0
[pid=28793] ppid=28737 vsize=377348 CPUtime=0.15 cores=0,2,4,6
/proc/28793/stat : 28793 (python) S 28737 28737 28639 0 -1 4202560 1168 0 0 0 9 6 0 0 19 0 2 0 538352810 386404352 7210 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/28793/statm: 94337 7210 400 364 0 30228 0
[pid=28795] ppid=28793 vsize=253028 CPUtime=100.08 cores=0,2,4,6
/proc/28795/stat : 28795 (FH) T 28793 28795 28795 0 -1 4202496 2501 0 0 0 10004 4 0 0 25 0 1 0 538352810 259100672 2232 33554432000 134512640 134936988 4286619552 18446744073709551615 134554594 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/28795/statm: 63257 2232 41 104 0 62761 0
[pid=28793/tid=28818] ppid=28737 vsize=377348 CPUtime=0 cores=0,2,4,6
/proc/28793/task/28818/stat : 28818 (python) S 28737 28737 28639 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538362824 386404352 7210 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604453926235072 0 0 -1 0 0 0 0
[pid=28797] ppid=28737 vsize=387844 CPUtime=0.45 cores=0,2,4,6
/proc/28797/stat : 28797 (python) S 28737 28737 28639 0 -1 4202560 1200 0 0 0 24 21 0 0 18 0 2 0 538352814 397152256 7214 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/28797/statm: 96961 7214 400 364 0 32849 0
[pid=28799] ppid=28797 vsize=268424 CPUtime=293.12 cores=0,2,4,6
/proc/28799/stat : 28799 (hybridGM3) T 28797 28799 28799 0 -1 4202496 6189 0 0 0 29311 1 0 0 25 0 1 0 538352815 274866176 5919 33554432000 134512640 135204385 4291746224 18446744073709551615 134538541 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/28799/statm: 67106 5919 63 169 0 66936 0
[pid=28797/tid=28834] ppid=28737 vsize=387844 CPUtime=0 cores=0,2,4,6
/proc/28797/task/28834/stat : 28834 (python) S 28737 28737 28639 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 538372358 397152256 7214 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604447488912064 0 0 -1 6 0 0 0
[pid=28800] ppid=28737 vsize=397876 CPUtime=0.15 cores=0,2,4,6
/proc/28800/stat : 28800 (python) S 28737 28737 28639 0 -1 4202560 1189 0 0 0 8 7 0 0 19 0 2 0 538352819 407425024 7218 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/28800/statm: 99469 7218 400 364 0 35354 0
[pid=28802] ppid=28800 vsize=1020308 CPUtime=98.43 cores=0,2,4,6
/proc/28802/stat : 28802 (gnovelty+2) T 28800 28802 28802 0 -1 4202496 1162 0 0 0 9843 0 0 0 25 0 1 0 538352819 1044795392 891 33554432000 134512640 135663704 4291134688 18446744073709551615 134544268 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/28802/statm: 255077 891 113 282 0 254793 0
[pid=28800/tid=28822] ppid=28737 vsize=397876 CPUtime=0 cores=0,2,4,6
/proc/28800/task/28822/stat : 28822 (python) S 28737 28737 28639 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 538362838 407425024 7218 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604448102495552 0 0 -1 4 0 0 0
[pid=28815] ppid=28737 vsize=408400 CPUtime=0.3 cores=0,2,4,6
/proc/28815/stat : 28815 (python) S 28737 28737 28639 0 -1 4202560 1194 0 0 0 16 14 0 0 18 0 2 0 538362722 418201600 7222 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/28815/statm: 102100 7222 400 364 0 37982 0
[pid=28817] ppid=28815 vsize=268424 CPUtime=197.93 cores=0,2,4,6
/proc/28817/stat : 28817 (hybridGM3) R 28815 28817 28817 0 -1 4202496 6186 0 0 0 19783 10 0 0 25 0 1 0 538362722 274866176 5918 33554432000 134512640 135204385 4286978896 18446744073709551615 134537699 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/28817/statm: 67106 5918 63 169 0 66936 0
[pid=28815/tid=28848] ppid=28737 vsize=408400 CPUtime=0 cores=0,2,4,6
/proc/28815/task/28848/stat : 28848 (python) S 28737 28737 28639 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 538382261 418201600 7222 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604453911221184 0 0 -1 2 0 0 0
[pid=28819] ppid=28737 vsize=418800 CPUtime=0.15 cores=0,2,4,6
/proc/28819/stat : 28819 (python) S 28737 28737 28639 0 -1 4202560 1177 0 0 0 8 7 0 0 19 0 2 0 538362829 428851200 7226 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/28819/statm: 104700 7226 400 364 0 40579 0
[pid=28821] ppid=28819 vsize=423400 CPUtime=98.29 cores=0,2,4,6
/proc/28821/stat : 28821 (adaptg2wsat2009) T 28819 28821 28821 0 -1 4202496 2630 0 0 0 9828 1 0 0 25 0 1 0 538362829 433561600 2358 33554432000 134512640 135107411 4286972992 18446744073709551615 134531561 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/28821/statm: 105850 2358 59 146 0 104726 0
[pid=28819/tid=28836] ppid=28737 vsize=418800 CPUtime=0 cores=0,2,4,6
/proc/28819/task/28836/stat : 28836 (python) S 28737 28737 28639 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 538372843 428851200 7226 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604453950476416 0 0 -1 4 0 0 0
[pid=28823] ppid=28737 vsize=428956 CPUtime=0.3 cores=0,2,4,6
/proc/28823/stat : 28823 (python) S 28737 28737 28639 0 -1 4202560 1226 0 0 0 15 15 0 0 19 0 2 0 538362843 439250944 7266 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/28823/statm: 107239 7266 408 364 0 43115 0
[pid=28825] ppid=28823 vsize=268424 CPUtime=196.33 cores=0,2,4,6
/proc/28825/stat : 28825 (hybridGM3) T 28823 28825 28825 0 -1 4202496 6196 0 0 0 19633 0 0 0 25 0 1 0 538362844 274866176 5919 33554432000 134512640 135204385 4291989344 18446744073709551615 134537704 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/28825/statm: 67106 5919 63 169 0 66936 0
[pid=28823/tid=28851] ppid=28737 vsize=428956 CPUtime=0 cores=0,2,4,6
/proc/28823/task/28851/stat : 28851 (python) S 28737 28737 28639 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 538382484 439250944 7266 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 514 18446604453857706304 0 0 -1 4 0 0 0
[pid=28837] ppid=28737 vsize=439244 CPUtime=0.13 cores=0,2,4,6
/proc/28837/stat : 28837 (python) S 28737 28737 28639 0 -1 4202560 1075 0 0 0 7 6 0 0 18 0 1 0 538372848 449785856 7256 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 808695200275726595 0 0 17 4 0 0 0
/proc/28837/statm: 109811 7256 380 364 0 45684 0
[pid=28839] ppid=28837 vsize=268424 CPUtime=96.4 cores=0,2,4,6
/proc/28839/stat : 28839 (hybridGM3) R 28837 28839 28839 0 -1 4202496 6193 0 0 0 9640 0 0 0 25 0 1 0 538372848 274866176 5919 33554432000 134512640 135204385 4292892640 18446744073709551615 134538541 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/28839/statm: 67106 5919 63 169 0 66936 0
[pid=28849] ppid=28737 vsize=449776 CPUtime=0 cores=0,2,4,6
/proc/28849/stat : 28849 (python) S 28737 28737 28639 0 -1 4202560 827 0 0 0 0 0 0 0 19 0 1 0 538382482 460570624 7236 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 3809933963498094591 0 0 17 0 0 0 0
/proc/28849/statm: 112444 7236 347 364 0 48314 0
[pid=28852] ppid=28849 vsize=454748 CPUtime=0.49 cores=0,2,4,6
/proc/28852/stat : 28852 (TNM) R 28849 28852 28852 0 -1 4202496 1494 0 0 0 49 0 0 0 21 0 1 0 538382484 465661952 1223 33554432000 134512640 135008748 4294460288 18446744073709551615 134523493 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/28852/statm: 113687 1223 45 122 0 112587 0
[pid=28853] ppid=28737 vsize=459952 CPUtime=0 cores=0,2,4,6
/proc/28853/stat : 28853 (python) S 28737 28737 28639 0 -1 4202560 839 0 0 0 0 0 0 0 21 0 1 0 538382489 470990848 7240 33554432000 4194304 5683644 140737046920928 18446744073709551615 210647166534 0 0 16781312 514 3809930115207397375 0 0 17 4 0 0 0
/proc/28853/statm: 114988 7240 347 364 0 50855 0
[pid=28855] ppid=28853 vsize=248616 CPUtime=0.41 cores=0,2,4,6
/proc/28855/stat : 28855 (FH) R 28853 28855 28855 0 -1 4202496 1485 0 0 0 41 0 0 0 25 0 1 0 538382489 254582784 1213 33554432000 134512640 134936988 4292225456 18446744073709551615 134523434 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/28855/statm: 62154 1213 37 104 0 61658 0
[pid=28737/tid=28745] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28745/stat : 28745 (python) S 28735 28737 28639 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 19 0 538332765 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 0 0 0 0
[pid=28737/tid=28748] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28748/stat : 28748 (python) S 28735 28737 28639 0 -1 4202560 22 935 0 0 0 0 0 0 20 0 19 0 538332768 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28751] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28751/stat : 28751 (python) S 28735 28737 28639 0 -1 4202560 22 935 0 0 0 0 0 0 22 0 19 0 538332771 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28754] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28754/stat : 28754 (python) S 28735 28737 28639 0 -1 4202560 23 935 0 0 0 0 0 0 24 0 19 0 538332774 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28765] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28765/stat : 28765 (python) S 28735 28737 28639 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 19 0 538342786 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28771] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28771/stat : 28771 (python) S 28735 28737 28639 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 19 0 538342790 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 2 0 0 0
[pid=28737/tid=28774] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28774/stat : 28774 (python) S 28735 28737 28639 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 19 0 538342793 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 0 0 0 0
[pid=28737/tid=28777] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28777/stat : 28777 (python) S 28735 28737 28639 0 -1 4202560 28 935 0 0 0 0 0 0 22 0 19 0 538342797 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 0 0 0 0
[pid=28737/tid=28789] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28789/stat : 28789 (python) S 28735 28737 28639 0 -1 4202560 32 935 0 0 0 0 0 0 20 0 19 0 538352704 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 0 0 0 0
[pid=28737/tid=28794] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28794/stat : 28794 (python) S 28735 28737 28639 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 19 0 538352810 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 0 0 0 0
[pid=28737/tid=28798] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28798/stat : 28798 (python) S 28735 28737 28639 0 -1 4202560 29 935 0 0 0 0 0 0 18 0 19 0 538352814 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 6 0 0 0
[pid=28737/tid=28801] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28801/stat : 28801 (python) S 28735 28737 28639 0 -1 4202560 33 935 0 0 0 0 0 0 20 0 19 0 538352819 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28816] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28816/stat : 28816 (python) S 28735 28737 28639 0 -1 4202560 33 935 0 0 0 0 0 0 18 0 19 0 538362722 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28820] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28820/stat : 28820 (python) S 28735 28737 28639 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 538362829 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28824] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28824/stat : 28824 (python) S 28735 28737 28639 0 -1 4202560 31 935 0 0 0 0 0 0 21 0 19 0 538362843 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28838] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28838/stat : 28838 (python) S 28735 28737 28639 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 19 0 538372848 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28850] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28850/stat : 28850 (python) S 28735 28737 28639 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 19 0 538382483 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
[pid=28737/tid=28854] ppid=28735 vsize=469644 CPUtime=0 cores=0,2,4,6
/proc/28737/task/28854/stat : 28854 (python) S 28735 28737 28639 0 -1 4202560 27 935 0 0 0 0 0 0 22 0 19 0 538382489 480915456 8808 33554432000 4194304 5683644 140737046920928 18446744073709551615 210658970033 0 0 16781312 2 18446604448102497344 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1984.32
Current children cumulated vsize (KiB) 16086216

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 28737 and gives
#  childrusage.ru_utime.tv_sec=3
#  childrusage.ru_utime.tv_usec=119525
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=204968
# CPU time returned by wait4() is 3.32449
# while last known CPU time is 1984.32
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 500.519
CPU time (s): 1984.32
CPU user time (s): 1982.64
CPU system time (s): 1.68
CPU usage (%): 396.453
Max. virtual memory (cumulated for all children) (KiB): 16086216

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3.11952
system time used= 0.204968
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 32055
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2924
involuntary context switches= 87

runsolver used 2.6046 second user time and 5.2632 second system time

The end

Launcher Data 

Begin job on node129 at 2011-05-01 18:28:48
IDJOB=3317110
IDBENCH=82575
IDSOLVER=1705
FILE ID=node129/3317110-1304267328
RUNJOBID= node129-1304267282-28657
PBS_JOBID= 13173950
Free space on /tmp= 67324 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k5-r20-v2000-c40000-S394483079-002-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317110-1304267328/watcher-3317110-1304267328 -o /tmp/evaluation-result-3317110-1304267328/solver-3317110-1304267328 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317110-1304267328.cnf -seed 2121985107 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 40e955d306a70bd3dffa8283c1221639
RANDOM SEED=2121985107

node129.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      30307652 kB
Buffers:        310832 kB
Cached:        1706408 kB
SwapCached:       5128 kB
Active:         535376 kB
Inactive:      1749100 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      30307652 kB
SwapTotal:    67111528 kB
SwapFree:     67080200 kB
Dirty:           39968 kB
Writeback:           0 kB
AnonPages:      264808 kB
Mapped:          18608 kB
Slab:           292468 kB
PageTables:       8568 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS:  1762304 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 67296 MiB
End job on node129 at 2011-05-01 18:37:11