Trace number 3317228

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 2392.16 603.921

General information on the benchmark

Namerandom/large/
unif-k5-r20-v1500-c30000-S64154096-097-UNKNOWN.cnf
MD5SUM1d861e510861ad58190eb0b3053bc657
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark340.927
Satisfiable
(Un)Satisfiability was proved
Number of variables1500
Number of clauses30000
Sum of the clauses size150000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 530000
Number of clauses of size over 50

Solver Data 

0.32/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.47	c INFO: solving HOME/instance-3317228-1304268405.cnf
0.32/0.47	c INFO: computing task features
0.50/0.63	c INFO: computed variable-clause graph statistics
0.50/0.64	c INFO: computed clause balance statistics
1.30/1.44	c INFO: computed variable balance statistics
1.70/1.87	c INFO: computed small-clause counts
1.70/1.87	c INFO: computed Horn-clause counts
1.80/1.90	c INFO: computed variable graph statistics
1.80/1.90	c INFO: collected features for HOME/instance-3317228-1304268405.cnf in 1.43 s
1.80/1.93	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.38)
1.80/1.96	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.14)
1.80/1.99	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.09)
1.90/2.02	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.07)
401.63/102.14	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.06)
401.63/102.18	c INFO: running gnovelty+2@0 for 97 with 4698 remaining (b = 0.05)
401.96/102.22	c INFO: running FH@0 for 97 with 4698 remaining (b = 0.04)
401.96/102.25	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.03)
800.95/202.34	c INFO: running adaptg2wsat2009++@0 for 97 with 4598 remaining (b = 0.03)
801.27/202.42	c INFO: running FH@0 for 97 with 4598 remaining (b = 0.02)
801.27/202.46	c INFO: running hybridGM3@0 for 194 with 4598 remaining (b = 0.04)
801.39/202.51	c INFO: running gnovelty+2@0 for 97 with 4598 remaining (b = 0.02)
1199.76/302.56	c INFO: running hybridGM3@0 for 194 with 4497 remaining (b = 0.03)
1200.09/302.64	c INFO: running hybridGM3@0 for 194 with 4497 remaining (b = 0.03)
1200.34/302.75	c INFO: running TNM@0 for 1552 with 4497 remaining (b = 0.21)
1583.43/398.91	c INFO: running adaptg2wsat2009++@0 for 97 with 4401 remaining (b = 0.02)
1981.12/499.02	c INFO: running hybridGM3@98 for 97 with 4301 remaining (b = 0.01)
1997.79/503.24	c INFO: running hybridGM3@99 for 97 with 4297 remaining (b = 0.01)
2001.34/504.14	c INFO: running hybridGM3@97 for 97 with 4296 remaining (b = 0.01)
2385.80/602.25	c INFO: running FH@0 for 97 with 4198 remaining (b = 0.01)
2390.92/603.50	c INFO: running hybridGM3@0 for 194 with 4197 remaining (b = 0.02)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317228-1304268405/watcher-3317228-1304268405 -o /tmp/evaluation-result-3317228-1304268405/solver-3317228-1304268405 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317228-1304268405.cnf -seed 966247080 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.47 8.04 5.68 5/255 30708
/proc/meminfo: memFree=30330012/32951132 swapFree=67111364/67111528
[pid=30708] ppid=30706 vsize=15772 CPUtime=0 cores=0,2,4,6
/proc/30708/stat : 30708 (python) R 30706 30708 30218 0 -1 4202496 859 0 0 0 0 0 0 0 25 0 1 0 538439154 16150528 741 33554432000 4194304 5683644 140733805563968 18446744073709551615 4553840 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/30708/statm: 3943 744 339 364 0 430 0

[startup+0.0893079 s]
/proc/loadavg: 8.47 8.04 5.68 5/255 30708
/proc/meminfo: memFree=30330012/32951132 swapFree=67111364/67111528
[pid=30708] ppid=30706 vsize=94176 CPUtime=0.06 cores=0,2,4,6
/proc/30708/stat : 30708 (python) R 30706 30708 30218 0 -1 4202496 2767 0 0 0 4 2 0 0 17 0 1 0 538439154 96436224 2474 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970593415 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/30708/statm: 23544 2474 764 364 0 1676 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 94176

[startup+0.100267 s]
/proc/loadavg: 8.47 8.04 5.68 5/255 30708
/proc/meminfo: memFree=30330012/32951132 swapFree=67111364/67111528
[pid=30708] ppid=30706 vsize=104216 CPUtime=0.07 cores=0,2,4,6
/proc/30708/stat : 30708 (python) S 30706 30708 30218 0 -1 4202496 3036 0 0 0 5 2 0 0 17 0 1 0 538439154 106717184 2725 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970548224 0 2147483391 16781312 2 18446744071699109054 0 0 17 0 0 0 0
/proc/30708/statm: 26054 2725 847 364 0 1802 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104216

[startup+0.300479 s]
/proc/loadavg: 8.47 8.04 5.68 5/255 30708
/proc/meminfo: memFree=30330012/32951132 swapFree=67111364/67111528
[pid=30708] ppid=30706 vsize=166544 CPUtime=0.22 cores=0,2,4,6
/proc/30708/stat : 30708 (python) S 30706 30708 30218 0 -1 4202496 6683 935 1 0 16 6 0 0 18 0 1 0 538439154 170541056 4772 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970548224 0 2147483391 16781312 2 18446744071699109054 0 0 17 6 0 0 0
/proc/30708/statm: 41636 4772 1322 364 0 3453 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 166544

[startup+0.700177 s]
/proc/loadavg: 8.47 8.04 5.68 5/255 30708
/proc/meminfo: memFree=30330012/32951132 swapFree=67111364/67111528
[pid=30708] ppid=30706 vsize=287588 CPUtime=0.6 cores=0,2,4,6
/proc/30708/stat : 30708 (python) R 30706 30708 30218 0 -1 4202496 12683 935 1 0 51 9 0 0 18 0 1 0 538439154 294490112 9282 33554432000 4194304 5683644 140733805563968 18446744073709551615 47912916368538 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/30708/statm: 71897 9282 1826 364 0 7821 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 287588

[startup+1.50105 s]
/proc/loadavg: 8.19 7.99 5.68 6/256 30713
/proc/meminfo: memFree=30298632/32951132 swapFree=67111364/67111528
[pid=30708] ppid=30706 vsize=287588 CPUtime=1.4 cores=0,2,4,6
/proc/30708/stat : 30708 (python) R 30706 30708 30218 0 -1 4202496 12684 935 1 0 131 9 0 0 19 0 1 0 538439154 294490112 9283 33554432000 4194304 5683644 140733805563968 18446744073709551615 5005552 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/30708/statm: 71897 9283 1827 364 0 7821 0
Current children cumulated CPU time (s) 1.4
Current children cumulated vsize (KiB) 287588

[startup+3.10073 s]
/proc/loadavg: 8.19 7.99 5.68 9/268 30725
/proc/meminfo: memFree=30274752/32951132 swapFree=67111364/67111528
[pid=30708] ppid=30706 vsize=324044 CPUtime=1.92 cores=0,2,4,6
/proc/30708/stat : 30708 (python) S 30706 30708 30218 0 -1 4202496 15633 935 1 0 182 10 0 0 20 0 5 0 538439154 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/30708/statm: 81011 8210 1883 364 0 16920 0
[pid=30714] ppid=30708 vsize=283516 CPUtime=0 cores=0,2,4,6
/proc/30714/stat : 30714 (python) S 30708 30708 30218 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 538439348 290320384 6584 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570255 0 0 16781312 514 2565372013006782723 0 0 17 2 0 0 0
/proc/30714/statm: 70879 6584 355 364 0 6797 0
[pid=30716] ppid=30714 vsize=1018996 CPUtime=1.15 cores=0,2,4,6
/proc/30716/stat : 30716 (gnovelty+2) R 30714 30716 30716 0 -1 4202496 961 0 0 0 115 0 0 0 25 0 1 0 538439348 1043451904 698 33554432000 134512640 135663704 4294688464 18446744073709551615 134546971 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/30716/statm: 254749 698 113 282 0 254465 0
[pid=30717] ppid=30708 vsize=293704 CPUtime=0 cores=0,2,4,6
/proc/30717/stat : 30717 (python) S 30708 30708 30218 0 -1 4202560 749 0 0 0 0 0 0 0 18 0 1 0 538439351 300752896 6665 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 11799495899191673092 0 0 17 2 0 0 0
/proc/30717/statm: 73426 6665 349 364 0 9341 0
[pid=30719] ppid=30717 vsize=1018996 CPUtime=1.11 cores=0,2,4,6
/proc/30719/stat : 30719 (gnovelty+2) R 30717 30719 30719 0 -1 4202496 960 0 0 0 111 0 0 0 25 0 1 0 538439351 1043451904 698 33554432000 134512640 135663704 4294066000 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30719/statm: 254749 698 113 282 0 254465 0
[pid=30720] ppid=30708 vsize=303960 CPUtime=0 cores=0,2,4,6
/proc/30720/stat : 30720 (python) S 30708 30708 30218 0 -1 4202560 736 0 0 0 0 0 0 0 18 0 1 0 538439354 311255040 6669 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 2565364316425420799 0 0 17 2 0 0 0
/proc/30720/statm: 75990 6669 349 364 0 11902 0
[pid=30722] ppid=30720 vsize=421428 CPUtime=1.09 cores=0,2,4,6
/proc/30722/stat : 30722 (adaptg2wsat2009) R 30720 30722 30722 0 -1 4202496 2049 0 0 0 109 0 0 0 25 0 1 0 538439354 431542272 1781 33554432000 134512640 135107411 4286970128 18446744073709551615 134531239 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30722/statm: 105357 1781 59 146 0 104233 0
[pid=30723] ppid=30708 vsize=314216 CPUtime=0 cores=0,2,4,6
/proc/30723/stat : 30723 (python) S 30708 30708 30218 0 -1 4202560 754 0 0 0 0 0 0 0 18 0 1 0 538439357 321757184 6673 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 5935651954192547839 0 0 17 0 0 0 0
/proc/30723/statm: 78554 6673 349 364 0 14463 0
[pid=30725] ppid=30723 vsize=421428 CPUtime=1.06 cores=0,2,4,6
/proc/30725/stat : 30725 (adaptg2wsat2009) R 30723 30725 30725 0 -1 4202496 2045 0 0 0 106 0 0 0 24 0 1 0 538439357 431542272 1781 33554432000 134512640 135107411 4288110208 18446744073709551615 134531613 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/30725/statm: 105357 1781 59 146 0 104233 0
[pid=30708/tid=30715] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30715/stat : 30715 (python) S 30706 30708 30218 0 -1 4202560 27 935 0 0 0 0 0 0 22 0 5 0 538439348 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 0 0 0 0
[pid=30708/tid=30718] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30718/stat : 30718 (python) S 30706 30708 30218 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 5 0 538439351 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 2 0 0 0
[pid=30708/tid=30721] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30721/stat : 30721 (python) S 30706 30708 30218 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 5 0 538439354 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 2 0 0 0
[pid=30708/tid=30724] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30724/stat : 30724 (python) S 30706 30708 30218 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 5 0 538439357 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 6.33
Current children cumulated vsize (KiB) 4400288

[startup+6.30118 s]
/proc/loadavg: 8.19 7.99 5.68 9/268 30725
/proc/meminfo: memFree=30267736/32951132 swapFree=67111364/67111528
[pid=30708] ppid=30706 vsize=324044 CPUtime=1.92 cores=0,2,4,6
/proc/30708/stat : 30708 (python) S 30706 30708 30218 0 -1 4202496 15633 935 1 0 182 10 0 0 20 0 5 0 538439154 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/30708/statm: 81011 8210 1883 364 0 16920 0
[pid=30714] ppid=30708 vsize=283516 CPUtime=0 cores=0,2,4,6
/proc/30714/stat : 30714 (python) S 30708 30708 30218 0 -1 4202560 899 0 0 0 0 0 0 0 18 0 1 0 538439348 290320384 6607 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570255 0 0 16781312 514 2565372013006782723 0 0 17 2 0 0 0
/proc/30714/statm: 70879 6607 378 364 0 6797 0
[pid=30716] ppid=30714 vsize=1018996 CPUtime=4.33 cores=0,2,4,6
/proc/30716/stat : 30716 (gnovelty+2) R 30714 30716 30716 0 -1 4202496 961 0 0 0 433 0 0 0 25 0 1 0 538439348 1043451904 698 33554432000 134512640 135663704 4294688464 18446744073709551615 135007724 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/30716/statm: 254749 698 113 282 0 254465 0
[pid=30717] ppid=30708 vsize=293704 CPUtime=0 cores=0,2,4,6
/proc/30717/stat : 30717 (python) S 30708 30708 30218 0 -1 4202560 918 0 0 0 0 0 0 0 18 0 1 0 538439351 300752896 6688 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 11799495899191673092 0 0 17 2 0 0 0
/proc/30717/statm: 73426 6688 372 364 0 9341 0
[pid=30719] ppid=30717 vsize=1018996 CPUtime=4.29 cores=0,2,4,6
/proc/30719/stat : 30719 (gnovelty+2) R 30717 30719 30719 0 -1 4202496 960 0 0 0 429 0 0 0 25 0 1 0 538439351 1043451904 698 33554432000 134512640 135663704 4294066000 18446744073709551615 134525309 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30719/statm: 254749 698 113 282 0 254465 0
[pid=30720] ppid=30708 vsize=303960 CPUtime=0 cores=0,2,4,6
/proc/30720/stat : 30720 (python) S 30708 30708 30218 0 -1 4202560 897 0 0 0 0 0 0 0 18 0 1 0 538439354 311255040 6691 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 2565364316425388289 0 0 17 2 0 0 0
/proc/30720/statm: 75990 6691 371 364 0 11902 0
[pid=30722] ppid=30720 vsize=421428 CPUtime=4.29 cores=0,2,4,6
/proc/30722/stat : 30722 (adaptg2wsat2009) R 30720 30722 30722 0 -1 4202496 2049 0 0 0 429 0 0 0 25 0 1 0 538439354 431542272 1781 33554432000 134512640 135107411 4286970128 18446744073709551615 134539228 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30722/statm: 105357 1781 59 146 0 104233 0
[pid=30723] ppid=30708 vsize=314216 CPUtime=0 cores=0,2,4,6
/proc/30723/stat : 30723 (python) S 30708 30708 30218 0 -1 4202560 925 0 0 0 0 0 0 0 18 0 1 0 538439357 321757184 6696 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 5935651954192515332 0 0 17 0 0 0 0
/proc/30723/statm: 78554 6696 372 364 0 14463 0
[pid=30725] ppid=30723 vsize=421428 CPUtime=4.23 cores=0,2,4,6
/proc/30725/stat : 30725 (adaptg2wsat2009) R 30723 30725 30725 0 -1 4202496 2045 0 0 0 423 0 0 0 25 0 1 0 538439357 431542272 1781 33554432000 134512640 135107411 4288110208 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/30725/statm: 105357 1781 59 146 0 104233 0
[pid=30708/tid=30715] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30715/stat : 30715 (python) S 30706 30708 30218 0 -1 4202560 27 935 0 0 0 0 0 0 22 0 5 0 538439348 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 0 0 0 0
[pid=30708/tid=30718] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30718/stat : 30718 (python) S 30706 30708 30218 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 5 0 538439351 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 2 0 0 0
[pid=30708/tid=30721] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30721/stat : 30721 (python) S 30706 30708 30218 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 5 0 538439354 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 2 0 0 0
[pid=30708/tid=30724] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30724/stat : 30724 (python) S 30706 30708 30218 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 5 0 538439357 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 19.06
Current children cumulated vsize (KiB) 4400288

[startup+12.7011 s]
/proc/loadavg: 8.55 8.07 5.73 9/268 30726
/proc/meminfo: memFree=30266872/32951132 swapFree=67111364/67111528
[pid=30708] ppid=30706 vsize=324044 CPUtime=1.92 cores=0,2,4,6
/proc/30708/stat : 30708 (python) S 30706 30708 30218 0 -1 4202496 15633 935 1 0 182 10 0 0 20 0 5 0 538439154 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/30708/statm: 81011 8210 1883 364 0 16920 0
[pid=30714] ppid=30708 vsize=283516 CPUtime=0.01 cores=0,2,4,6
/proc/30714/stat : 30714 (python) S 30708 30708 30218 0 -1 4202560 902 0 0 0 1 0 0 0 18 0 1 0 538439348 290320384 6607 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570255 0 0 16781312 514 10758606928578838787 0 0 17 2 0 0 0
/proc/30714/statm: 70879 6607 378 364 0 6797 0
[pid=30716] ppid=30714 vsize=1018996 CPUtime=10.73 cores=0,2,4,6
/proc/30716/stat : 30716 (gnovelty+2) R 30714 30716 30716 0 -1 4202496 961 0 0 0 1072 1 0 0 25 0 1 0 538439348 1043451904 698 33554432000 134512640 135663704 4294688464 18446744073709551615 134524744 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/30716/statm: 254749 698 113 282 0 254465 0
[pid=30717] ppid=30708 vsize=293704 CPUtime=0.01 cores=0,2,4,6
/proc/30717/stat : 30717 (python) S 30708 30708 30218 0 -1 4202560 921 0 0 0 0 1 0 0 18 0 1 0 538439351 300752896 6688 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 10758606928578838788 0 0 17 2 0 0 0
/proc/30717/statm: 73426 6688 372 364 0 9341 0
[pid=30719] ppid=30717 vsize=1018996 CPUtime=10.64 cores=0,2,4,6
/proc/30719/stat : 30719 (gnovelty+2) R 30717 30719 30719 0 -1 4202496 960 0 0 0 1063 1 0 0 25 0 1 0 538439351 1043451904 698 33554432000 134512640 135663704 4294066000 18446744073709551615 134544240 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30719/statm: 254749 698 113 282 0 254465 0
[pid=30720] ppid=30708 vsize=303960 CPUtime=0.01 cores=0,2,4,6
/proc/30720/stat : 30720 (python) S 30708 30708 30218 0 -1 4202560 900 0 0 0 1 0 0 0 18 0 1 0 538439354 311255040 6691 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 10758606928578838785 0 0 17 2 0 0 0
/proc/30720/statm: 75990 6691 371 364 0 11902 0
[pid=30722] ppid=30720 vsize=421428 CPUtime=10.69 cores=0,2,4,6
/proc/30722/stat : 30722 (adaptg2wsat2009) R 30720 30722 30722 0 -1 4202496 2049 0 0 0 1068 1 0 0 25 0 1 0 538439354 431542272 1781 33554432000 134512640 135107411 4286970128 18446744073709551615 134531561 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30722/statm: 105357 1781 59 146 0 104233 0
[pid=30723] ppid=30708 vsize=314216 CPUtime=0.01 cores=0,2,4,6
/proc/30723/stat : 30723 (python) S 30708 30708 30218 0 -1 4202560 926 0 0 0 1 0 0 0 18 0 1 0 538439357 321757184 6696 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 10758606928578838788 0 0 17 0 0 0 0
/proc/30723/statm: 78554 6696 372 364 0 14463 0
[pid=30725] ppid=30723 vsize=421428 CPUtime=10.6 cores=0,2,4,6
/proc/30725/stat : 30725 (adaptg2wsat2009) R 30723 30725 30725 0 -1 4202496 2045 0 0 0 1060 0 0 0 25 0 1 0 538439357 431542272 1781 33554432000 134512640 135107411 4288110208 18446744073709551615 134531720 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/30725/statm: 105357 1781 59 146 0 104233 0
[pid=30708/tid=30715] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30715/stat : 30715 (python) S 30706 30708 30218 0 -1 4202560 27 935 0 0 0 0 0 0 22 0 5 0 538439348 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 0 0 0 0
[pid=30708/tid=30718] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30718/stat : 30718 (python) S 30706 30708 30218 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 5 0 538439351 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 2 0 0 0
[pid=30708/tid=30721] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30721/stat : 30721 (python) S 30706 30708 30218 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 5 0 538439354 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 2 0 0 0
[pid=30708/tid=30724] ppid=30706 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30724/stat : 30724 (python) S 30706 30708 30218 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 5 0 538439357 331821056 8210 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 44.62
Current children cumulated vsize (KiB) 4400288

[startup+25.5009 s]
/proc/loadavg: 8.46 8.07 5.75 9/268 30726
/proc/meminfo: memFree=30267188/32951132 swapFree=67111364/67111528
[pid=30708] ppid=30706 vsize=324044 CPUtime=1.92 cores=0,2,4,6

################
# More data... #
################

/proc/30723/stat : 30723 (python) S 30708 30708 30218 0 -1 4202560 1118 0 0 0 9 6 0 0 19 0 2 0 538439357 321757184 6725 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30723/statm: 78554 6725 400 364 0 14463 0
[pid=30725] ppid=30723 vsize=421428 CPUtime=99.68 cores=0,2,4,6
/proc/30725/stat : 30725 (adaptg2wsat2009) T 30723 30725 30725 0 -1 4202496 2045 0 0 0 9967 1 0 0 25 0 1 0 538439357 431542272 1781 33554432000 134512640 135107411 4288110208 18446744073709551615 134531709 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30725/statm: 105357 1781 59 146 0 104233 0
[pid=30723/tid=30785] ppid=30708 vsize=314216 CPUtime=0 cores=0,2,4,6
/proc/30723/task/30785/stat : 30785 (python) S 30708 30708 30218 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538449371 321757184 6725 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604449999727616 0 0 -1 0 0 0 0
[pid=30781] ppid=30708 vsize=324472 CPUtime=0.16 cores=0,2,4,6
/proc/30781/stat : 30781 (python) S 30708 30708 30218 0 -1 4202560 1116 0 0 0 8 8 0 0 19 0 2 0 538449369 332259328 6729 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/30781/statm: 81118 6729 400 364 0 17024 0
[pid=30784] ppid=30781 vsize=421428 CPUtime=99.97 cores=0,2,4,6
/proc/30784/stat : 30784 (adaptg2wsat2009) T 30781 30784 30784 0 -1 4202496 2048 0 0 0 9997 0 0 0 25 0 1 0 538449369 431542272 1783 33554432000 134512640 135107411 4292989840 18446744073709551615 134539270 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/30784/statm: 105357 1783 59 146 0 104233 0
[pid=30781/tid=30814] ppid=30708 vsize=324472 CPUtime=0 cores=0,2,4,6
/proc/30781/task/30814/stat : 30814 (python) S 30708 30708 30218 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538459384 332259328 6729 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604450658281280 0 0 -1 2 0 0 0
[pid=30786] ppid=30708 vsize=334812 CPUtime=0.16 cores=0,2,4,6
/proc/30786/stat : 30786 (python) S 30708 30708 30218 0 -1 4202560 1134 0 0 0 7 9 0 0 19 0 2 0 538449372 342847488 6733 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/30786/statm: 83703 6733 400 364 0 19606 0
[pid=30788] ppid=30786 vsize=1018996 CPUtime=100.14 cores=0,2,4,6
/proc/30788/stat : 30788 (gnovelty+2) T 30786 30788 30788 0 -1 4202496 958 0 0 0 10010 4 0 0 25 0 1 0 538449373 1043451904 698 33554432000 134512640 135663704 4289945584 18446744073709551615 134524548 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30788/statm: 254749 698 113 282 0 254465 0
[pid=30786/tid=30819] ppid=30708 vsize=334812 CPUtime=0 cores=0,2,4,6
/proc/30786/task/30819/stat : 30819 (python) S 30708 30708 30218 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 538459393 342847488 6733 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604446547844160 0 0 -1 4 0 0 0
[pid=30789] ppid=30708 vsize=345248 CPUtime=0.15 cores=0,2,4,6
/proc/30789/stat : 30789 (python) S 30708 30708 30218 0 -1 4202560 1195 0 0 0 8 7 0 0 19 0 2 0 538449376 353533952 6747 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30789/statm: 86312 6747 406 364 0 22212 0
[pid=30791] ppid=30789 vsize=251292 CPUtime=99.98 cores=0,2,4,6
/proc/30791/stat : 30791 (FH) T 30789 30791 30791 0 -1 4202496 1945 0 0 0 9998 0 0 0 25 0 1 0 538449376 257323008 1681 33554432000 134512640 134936988 4293097920 18446744073709551615 134545096 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/30791/statm: 62823 1681 41 104 0 62327 0
[pid=30789/tid=30818] ppid=30708 vsize=345248 CPUtime=0 cores=0,2,4,6
/proc/30789/task/30818/stat : 30818 (python) S 30708 30708 30218 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538459391 353533952 6747 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604450658279488 0 0 -1 0 0 0 0
[pid=30792] ppid=30708 vsize=355324 CPUtime=0.15 cores=0,2,4,6
/proc/30792/stat : 30792 (python) S 30708 30708 30218 0 -1 4202560 1185 0 0 0 5 10 0 0 19 0 2 0 538449380 363851776 6796 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/30792/statm: 88831 6796 406 364 0 24728 0
[pid=30794] ppid=30792 vsize=421428 CPUtime=98.57 cores=0,2,4,6
/proc/30794/stat : 30794 (adaptg2wsat2009) T 30792 30794 30794 0 -1 4202496 2052 0 0 0 9855 2 0 0 25 0 1 0 538449380 431542272 1783 33554432000 134512640 135107411 4289261600 18446744073709551615 134531792 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/30794/statm: 105357 1783 59 146 0 104233 0
[pid=30792/tid=30822] ppid=30708 vsize=355324 CPUtime=0 cores=0,2,4,6
/proc/30792/task/30822/stat : 30822 (python) S 30708 30708 30218 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538459397 363851776 6796 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604453715994688 0 0 -1 6 0 0 0
[pid=30815] ppid=30708 vsize=365788 CPUtime=0.17 cores=0,2,4,6
/proc/30815/stat : 30815 (python) S 30708 30708 30218 0 -1 4202560 1159 0 0 0 9 8 0 0 19 0 2 0 538459389 374566912 6794 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/30815/statm: 91447 6794 400 364 0 27341 0
[pid=30817] ppid=30815 vsize=421428 CPUtime=100.16 cores=0,2,4,6
/proc/30817/stat : 30817 (adaptg2wsat2009) T 30815 30817 30817 0 -1 4202496 2053 0 0 0 10016 0 0 0 25 0 1 0 538459389 431542272 1781 33554432000 134512640 135107411 4294482816 18446744073709551615 134532242 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/30817/statm: 105357 1781 59 146 0 104233 0
[pid=30815/tid=30848] ppid=30708 vsize=365788 CPUtime=0 cores=0,2,4,6
/proc/30815/task/30848/stat : 30848 (python) S 30708 30708 30218 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 538469406 374566912 6794 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604451850769600 0 0 -1 6 0 0 0
[pid=30820] ppid=30708 vsize=376156 CPUtime=0.16 cores=0,2,4,6
/proc/30820/stat : 30820 (python) S 30708 30708 30218 0 -1 4202560 1176 0 0 0 9 7 0 0 19 0 2 0 538459396 385183744 6805 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/30820/statm: 94039 6805 400 364 0 29930 0
[pid=30823] ppid=30820 vsize=251292 CPUtime=99.91 cores=0,2,4,6
/proc/30823/stat : 30823 (FH) T 30820 30823 30823 0 -1 4202496 1952 0 0 0 9987 4 0 0 25 0 1 0 538459397 257323008 1682 33554432000 134512640 134936988 4288569360 18446744073709551615 134554083 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/30823/statm: 62823 1682 41 104 0 62327 0
[pid=30820/tid=30852] ppid=30708 vsize=376156 CPUtime=0 cores=0,2,4,6
/proc/30820/task/30852/stat : 30852 (python) S 30708 30708 30218 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538469413 385183744 6805 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604453866630208 0 0 -1 6 0 0 0
[pid=30824] ppid=30708 vsize=386532 CPUtime=0.54 cores=0,2,4,6
/proc/30824/stat : 30824 (python) S 30708 30708 30218 0 -1 4202560 1201 0 0 0 29 25 0 0 18 0 2 0 538459400 395808768 6837 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/30824/statm: 96633 6837 400 364 0 32521 0
[pid=30826] ppid=30824 vsize=268428 CPUtime=295.52 cores=0,2,4,6
/proc/30826/stat : 30826 (hybridGM3) T 30824 30826 30826 0 -1 4202496 5811 0 0 0 29543 9 0 0 25 0 1 0 538459401 274870272 5544 33554432000 134512640 135204385 4288784656 18446744073709551615 134538524 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30826/statm: 67107 5544 63 169 0 66937 0
[pid=30824/tid=30875] ppid=30708 vsize=386532 CPUtime=0 cores=0,2,4,6
/proc/30824/task/30875/stat : 30875 (python) S 30708 30708 30218 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 538479040 395808768 6837 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604449866830272 0 0 -1 6 0 0 0
[pid=30827] ppid=30708 vsize=396916 CPUtime=0.17 cores=0,2,4,6
/proc/30827/stat : 30827 (python) S 30708 30708 30218 0 -1 4202560 1180 0 0 0 9 8 0 0 19 0 2 0 538459405 406441984 6871 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/30827/statm: 99229 6871 400 364 0 35114 0
[pid=30829] ppid=30827 vsize=1018996 CPUtime=97.92 cores=0,2,4,6
/proc/30829/stat : 30829 (gnovelty+2) T 30827 30829 30829 0 -1 4202496 966 0 0 0 9792 0 0 0 25 0 1 0 538459405 1043451904 697 33554432000 134512640 135663704 4292000752 18446744073709551615 134525684 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30829/statm: 254749 697 113 282 0 254465 0
[pid=30827/tid=30856] ppid=30708 vsize=396916 CPUtime=0 cores=0,2,4,6
/proc/30827/task/30856/stat : 30856 (python) S 30708 30708 30218 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 538469425 406441984 6871 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604449866831168 0 0 -1 4 0 0 0
[pid=30849] ppid=30708 vsize=406984 CPUtime=0.81 cores=0,2,4,6
/proc/30849/stat : 30849 (python) S 30708 30708 30218 0 -1 4202560 1200 0 0 0 49 32 0 0 18 0 2 0 538469410 416751616 6907 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/30849/statm: 101746 6907 400 364 0 37628 0
[pid=30851] ppid=30849 vsize=268424 CPUtime=294.01 cores=0,2,4,6
/proc/30851/stat : 30851 (hybridGM3) R 30849 30851 30851 0 -1 4202496 5809 0 0 0 29388 13 0 0 25 0 1 0 538469411 274866176 5544 33554432000 134512640 135204385 4289453920 18446744073709551615 134537949 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/30851/statm: 67106 5544 63 169 0 66936 0
[pid=30849/tid=30887] ppid=30708 vsize=406984 CPUtime=0 cores=0,2,4,6
/proc/30849/task/30887/stat : 30887 (python) S 30708 30708 30218 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538489051 416751616 6907 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604450441995392 0 0 -1 2 0 0 0
[pid=30853] ppid=30708 vsize=417384 CPUtime=0.57 cores=0,2,4,6
/proc/30853/stat : 30853 (python) S 30708 30708 30218 0 -1 4202560 1214 0 0 0 32 25 0 0 18 0 2 0 538469418 427401216 6911 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30853/statm: 104346 6911 400 364 0 40225 0
[pid=30855] ppid=30853 vsize=268424 CPUtime=295.92 cores=0,2,4,6
/proc/30855/stat : 30855 (hybridGM3) T 30853 30855 30855 0 -1 4202496 5812 0 0 0 29576 16 0 0 25 0 1 0 538469418 274866176 5543 33554432000 134512640 135204385 4289956816 18446744073709551615 134537911 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/30855/statm: 67106 5543 63 169 0 66936 0
[pid=30853/tid=30889] ppid=30708 vsize=417384 CPUtime=0 cores=0,2,4,6
/proc/30853/task/30889/stat : 30889 (python) S 30708 30708 30218 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 538489564 427401216 6911 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604450441996288 0 0 -1 4 0 0 0
[pid=30857] ppid=30708 vsize=427792 CPUtime=1.27 cores=0,2,4,6
/proc/30857/stat : 30857 (python) S 30708 30708 30218 0 -1 4202560 1089 0 0 0 60 67 0 0 18 0 1 0 538469430 438059008 6893 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 10758606928578838785 0 0 17 0 0 0 0
/proc/30857/statm: 106948 6893 379 364 0 42824 0
[pid=30859] ppid=30857 vsize=459764 CPUtime=299.75 cores=0,2,4,6
/proc/30859/stat : 30859 (TNM) R 30857 30859 30859 0 -1 4202496 1930 0 0 0 29970 5 0 0 25 0 1 0 538469430 470798336 1652 33554432000 134512640 135008748 4287435584 18446744073709551615 134528982 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/30859/statm: 114941 1652 46 122 0 113841 0
[pid=30876] ppid=30708 vsize=437956 CPUtime=0.19 cores=0,2,4,6
/proc/30876/stat : 30876 (python) S 30708 30708 30218 0 -1 4202560 1189 0 0 0 10 9 0 0 19 0 2 0 538479045 448466944 6954 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982376907 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/30876/statm: 109489 6954 408 364 0 45362 0
[pid=30878] ppid=30876 vsize=421428 CPUtime=101.43 cores=0,2,4,6
/proc/30878/stat : 30878 (adaptg2wsat2009) T 30876 30878 30878 0 -1 4202496 2057 0 0 0 10141 2 0 0 25 0 1 0 538479046 431542272 1781 33554432000 134512640 135107411 4288027712 18446744073709551615 134531817 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30878/statm: 105357 1781 59 146 0 104233 0
[pid=30876/tid=30888] ppid=30708 vsize=437956 CPUtime=0 cores=0,2,4,6
/proc/30876/task/30888/stat : 30888 (python) S 30708 30708 30218 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 538489473 448466944 6954 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 514 18446604443076875200 0 0 -1 6 0 0 0
[pid=30946] ppid=30708 vsize=448380 CPUtime=0 cores=0,2,4,6
/proc/30946/stat : 30946 (python) S 30708 30708 30218 0 -1 4202560 851 0 0 0 0 0 0 0 18 0 1 0 538499379 459141120 6918 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 11799488202610311167 0 0 17 4 0 0 0
/proc/30946/statm: 112095 6918 351 364 0 47965 0
[pid=30948] ppid=30946 vsize=251292 CPUtime=1.55 cores=0,2,4,6
/proc/30948/stat : 30948 (FH) R 30946 30948 30948 0 -1 4202496 1958 0 0 0 155 0 0 0 25 0 1 0 538499380 257323008 1681 33554432000 134512640 134936988 4286810784 18446744073709551615 134554229 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30948/statm: 62823 1681 41 104 0 62327 0
[pid=30949] ppid=30708 vsize=458552 CPUtime=0 cores=0,2,4,6
/proc/30949/stat : 30949 (python) S 30708 30708 30218 0 -1 4202560 861 0 0 0 0 0 0 0 19 0 1 0 538499505 469557248 6922 33554432000 4194304 5683644 140733805563968 18446744073709551615 242970570310 0 0 16781312 514 4955160261100044287 0 0 17 4 0 0 0
/proc/30949/statm: 114638 6922 351 364 0 50505 0
[pid=30951] ppid=30949 vsize=268424 CPUtime=0.27 cores=0,2,4,6
/proc/30951/stat : 30951 (hybridGM3) R 30949 30951 30951 0 -1 4202496 5816 0 0 0 27 0 0 0 21 0 1 0 538499505 274866176 5544 33554432000 134512640 135204385 4293444576 18446744073709551615 134538282 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30951/statm: 67106 5544 63 169 0 66936 0
[pid=30708/tid=30715] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30715/stat : 30715 (python) S 30706 30708 30218 0 -1 4202560 27 935 0 0 0 0 0 0 22 0 19 0 538439348 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 0 0 0 0
[pid=30708/tid=30718] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30718/stat : 30718 (python) S 30706 30708 30218 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 19 0 538439351 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 2 0 0 0
[pid=30708/tid=30721] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30721/stat : 30721 (python) S 30706 30708 30218 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 19 0 538439354 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 2 0 0 0
[pid=30708/tid=30724] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30724/stat : 30724 (python) S 30706 30708 30218 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 19 0 538439357 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 0 0 0 0
[pid=30708/tid=30783] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30783/stat : 30783 (python) S 30706 30708 30218 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 19 0 538449369 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 6 0 0 0
[pid=30708/tid=30787] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30787/stat : 30787 (python) S 30706 30708 30218 0 -1 4202560 24 935 0 0 0 0 0 0 22 0 19 0 538449372 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 4 0 0 0
[pid=30708/tid=30790] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30790/stat : 30790 (python) S 30706 30708 30218 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 19 0 538449376 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 0 0 0 0
[pid=30708/tid=30793] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30793/stat : 30793 (python) S 30706 30708 30218 0 -1 4202560 29 935 0 0 0 0 0 0 22 0 19 0 538449380 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 6 0 0 0
[pid=30708/tid=30816] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30816/stat : 30816 (python) S 30706 30708 30218 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 19 0 538459389 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 6 0 0 0
[pid=30708/tid=30821] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30821/stat : 30821 (python) S 30706 30708 30218 0 -1 4202560 23 935 0 0 0 0 0 0 21 0 19 0 538459396 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 0 0 0 0
[pid=30708/tid=30825] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30825/stat : 30825 (python) S 30706 30708 30218 0 -1 4202560 28 935 0 0 0 0 0 0 18 0 19 0 538459401 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 6 0 0 0
[pid=30708/tid=30828] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30828/stat : 30828 (python) S 30706 30708 30218 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 19 0 538459405 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 4 0 0 0
[pid=30708/tid=30850] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30850/stat : 30850 (python) S 30706 30708 30218 0 -1 4202560 33 935 0 0 0 0 0 0 18 0 19 0 538469410 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 4 0 0 0
[pid=30708/tid=30854] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30854/stat : 30854 (python) S 30706 30708 30218 0 -1 4202560 32 935 0 0 0 0 0 0 18 0 19 0 538469418 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 4 0 0 0
[pid=30708/tid=30858] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30858/stat : 30858 (python) S 30706 30708 30218 0 -1 4202560 24 935 0 0 0 0 0 0 22 0 19 0 538469430 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 4 0 0 0
[pid=30708/tid=30877] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30877/stat : 30877 (python) S 30706 30708 30218 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 538479045 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 4 0 0 0
[pid=30708/tid=30947] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30947/stat : 30947 (python) S 30706 30708 30218 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 19 0 538499380 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 4 0 0 0
[pid=30708/tid=30950] ppid=30706 vsize=468392 CPUtime=0 cores=0,2,4,6
/proc/30708/task/30950/stat : 30950 (python) S 30706 30708 30218 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 19 0 538499505 479633408 8486 33554432000 4194304 5683644 140733805563968 18446744073709551615 242982373809 0 0 16781312 2 18446604453715995584 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2392.16
Current children cumulated vsize (KiB) 16048220

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 30708 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=544613
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=154976
# CPU time returned by wait4() is 2.69959
# while last known CPU time is 2392.16
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 603.921
CPU time (s): 2392.16
CPU user time (s): 2388.99
CPU system time (s): 3.17
CPU usage (%): 396.105
Max. virtual memory (cumulated for all children) (KiB): 16048220

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.54461
system time used= 0.154976
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 31145
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1133
involuntary context switches= 74

runsolver used 4.06638 second user time and 6.79997 second system time

The end

Launcher Data 

Begin job on node128 at 2011-05-01 18:46:45
IDJOB=3317228
IDBENCH=82693
IDSOLVER=1705
FILE ID=node128/3317228-1304268405
RUNJOBID= node128-1304267283-30236
PBS_JOBID= 13173951
Free space on /tmp= 67600 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k5-r20-v1500-c30000-S64154096-097-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317228-1304268405/watcher-3317228-1304268405 -o /tmp/evaluation-result-3317228-1304268405/solver-3317228-1304268405 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317228-1304268405.cnf -seed 966247080 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1d861e510861ad58190eb0b3053bc657
RANDOM SEED=966247080

node128.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5434.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      30330292 kB
Buffers:        490012 kB
Cached:        1267064 kB
SwapCached:          0 kB
Active:         695416 kB
Inactive:      1483860 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      30330292 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:           33004 kB
Writeback:           0 kB
AnonPages:      421900 kB
Mapped:          23340 kB
Slab:           362512 kB
PageTables:      19552 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS: 18560312 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 67552 MiB
End job on node128 at 2011-05-01 18:56:51