Trace number 3378133

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 1978.95 499.455

General information on the benchmark

Namerandom/large/
unif-k7-r85-v300-c25500-S942666749-061-UNKNOWN.cnf
MD5SUM456ae67a44e812130272597099547869
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables300
Number of clauses25500
Sum of the clauses size178500
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 525500

Solver Data 

0.37/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.37/0.67	c INFO: solving HOME/instance-3378133-1305463018.cnf
0.37/0.67	c INFO: computing task features
0.54/0.84	c INFO: computed variable-clause graph statistics
0.54/0.84	c INFO: computed clause balance statistics
1.45/1.77	c INFO: computed variable balance statistics
1.85/2.14	c INFO: computed small-clause counts
1.85/2.14	c INFO: computed Horn-clause counts
1.85/2.17	c INFO: computed variable graph statistics
1.85/2.17	c INFO: collected features for HOME/instance-3378133-1305463018.cnf in 1.49 s
1.85/2.18	c INFO: running gnovelty+2@0 for 97 with 1198 remaining (b = 0.37)
1.85/2.19	c INFO: running gnovelty+2@0 for 97 with 1198 remaining (b = 0.13)
1.85/2.20	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.08)
1.95/2.21	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.07)
393.87/100.55	c INFO: running adaptg2wsat2009++@0 for 97 with 1100 remaining (b = 0.05)
401.43/102.48	c INFO: running gnovelty+2@0 for 97 with 1098 remaining (b = 0.04)
401.84/102.51	c INFO: running FH@0 for 97 with 1098 remaining (b = 0.03)
401.84/102.52	c INFO: running adaptg2wsat2009++@0 for 97 with 1098 remaining (b = 0.03)
792.86/200.83	c INFO: running adaptg2wsat2009++@0 for 97 with 999 remaining (b = 0.02)
800.80/202.80	c INFO: running FH@0 for 97 with 997 remaining (b = 0.02)
800.80/202.81	c INFO: running hybridGM3@0 for 194 with 997 remaining (b = 0.04)
800.80/202.83	c INFO: running gnovelty+2@0 for 97 with 997 remaining (b = 0.02)
1191.71/301.13	c INFO: running hybridGM3@0 for 194 with 899 remaining (b = 0.03)
1195.69/302.18	c INFO: running adaptg2wsat2009++@0 for 97 with 898 remaining (b = 0.01)
1199.26/303.10	c INFO: running hybridGM3@0 for 194 with 897 remaining (b = 0.03)
1574.58/397.50	c INFO: running hybridGM3@100 for 97 with 803 remaining (b = 0.01)
1590.91/401.52	c INFO: running hybridGM3@0 for 194 with 799 remaining (b = 0.02)
1965.01/495.85	c INFO: running TNM@0 for 704 with 704 remaining (b = 0.08)
1976.50/498.79	c INFO: running FH@0 for 97 with 701 remaining (b = 0.01)
1978.95/499.41	c INFO: running adaptg2wsat2009++@0 for 97 with 701 remaining (b = 0.01)
1978.95/499.41	Traceback (most recent call last):
1978.95/499.41	  File "HOME/solve", line 33, in <module>
1978.95/499.41	    plac.call(main)
1978.95/499.41	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378133-1305463018/watcher-3378133-1305463018 -o /tmp/evaluation-result-3378133-1305463018/solver-3378133-1305463018 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378133-1305463018.cnf -seed 570989988 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.68 3.77 3.75 6/375 2226
/proc/meminfo: memFree=26694804/32950928 swapFree=60199044/67111528
[pid=2226] ppid=2224 vsize=15080 CPUtime=0 cores=1,3,5,7
/proc/2226/stat : 2226 (python) R 2224 2226 2082 0 -1 4202496 649 0 0 0 0 0 0 0 25 0 1 0 657900667 15441920 545 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272687104 0 2147483391 16781312 2 0 0 0 17 5 0 0 0
/proc/2226/statm: 3804 550 312 364 0 291 0

[startup+0.0722649 s]
/proc/loadavg: 5.68 3.77 3.75 6/375 2226
/proc/meminfo: memFree=26694804/32950928 swapFree=60199044/67111528
[pid=2226] ppid=2224 vsize=36508 CPUtime=0.04 cores=1,3,5,7
/proc/2226/stat : 2226 (python) S 2224 2226 2082 0 -1 4202496 1613 0 0 0 3 1 0 0 25 0 1 0 657900667 37384192 1445 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272687104 0 2147483391 16781312 2 18446744071699199166 0 0 17 5 0 0 0
/proc/2226/statm: 9127 1445 452 364 0 944 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 36508

[startup+0.100258 s]
/proc/loadavg: 5.68 3.77 3.75 6/375 2226
/proc/meminfo: memFree=26694804/32950928 swapFree=60199044/67111528
[pid=2226] ppid=2224 vsize=85620 CPUtime=0.06 cores=1,3,5,7
/proc/2226/stat : 2226 (python) R 2224 2226 2082 0 -1 4202496 2630 0 0 0 5 1 0 0 25 0 1 0 657900667 87674880 2348 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272687104 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/2226/statm: 21405 2348 713 364 0 1622 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85620

[startup+0.300288 s]
/proc/loadavg: 5.68 3.77 3.75 6/375 2226
/proc/meminfo: memFree=26694804/32950928 swapFree=60199044/67111528
[pid=2226] ppid=2224 vsize=135748 CPUtime=0.19 cores=1,3,5,7
/proc/2226/stat : 2226 (python) R 2224 2226 2082 0 -1 4202496 5963 932 1 0 14 5 0 0 17 0 1 0 657900667 139005952 4151 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272685061 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/2226/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135748

[startup+0.700151 s]
/proc/loadavg: 5.68 3.77 3.75 6/375 2226
/proc/meminfo: memFree=26694804/32950928 swapFree=60199044/67111528
[pid=2226] ppid=2224 vsize=279800 CPUtime=0.44 cores=1,3,5,7
/proc/2226/stat : 2226 (python) R 2224 2226 2082 0 -1 4202496 9610 932 1 0 33 11 0 0 15 0 1 0 657900667 286515200 7293 33554432000 4194304 5683644 140735438242800 18446744073709551615 4960197 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/2226/statm: 69950 7293 1761 364 0 5874 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 279800

[startup+1.50101 s]
/proc/loadavg: 5.68 3.77 3.75 6/376 2231
/proc/meminfo: memFree=26664872/32950928 swapFree=60199044/67111528
[pid=2226] ppid=2224 vsize=285636 CPUtime=1.25 cores=1,3,5,7
/proc/2226/stat : 2226 (python) R 2224 2226 2082 0 -1 4202496 12497 932 1 0 113 12 0 0 16 0 1 0 657900667 292491264 8794 33554432000 4194304 5683644 140735438242800 18446744073709551615 5003260 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/2226/statm: 71409 8794 1826 364 0 7333 0
Current children cumulated CPU time (s) 1.25
Current children cumulated vsize (KiB) 285636

[startup+3.10073 s]
/proc/loadavg: 5.68 3.77 3.75 6/376 2231
/proc/meminfo: memFree=26664252/32950928 swapFree=60199044/67111528
[pid=2226] ppid=2224 vsize=321288 CPUtime=1.96 cores=1,3,5,7
/proc/2226/stat : 2226 (python) S 2224 2226 2082 0 -1 4202496 15256 932 1 0 183 13 0 0 25 0 5 0 657900667 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/2226/statm: 80322 7530 1875 364 0 16231 0
Current children cumulated CPU time (s) 1.96
Current children cumulated vsize (KiB) 321288
heavy processes:

[startup+6.30116 s]
/proc/loadavg: 5.95 3.86 3.78 9/388 2252
/proc/meminfo: memFree=26640340/32950928 swapFree=60199044/67111528
[pid=2226] ppid=2224 vsize=321288 CPUtime=1.96 cores=1,3,5,7
/proc/2226/stat : 2226 (python) S 2224 2226 2082 0 -1 4202496 15256 932 1 0 183 13 0 0 25 0 5 0 657900667 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/2226/statm: 80322 7530 1875 364 0 16231 0
[pid=2232] ppid=2226 vsize=280760 CPUtime=0.01 cores=1,3,5,7
/proc/2232/stat : 2232 (python) S 2226 2226 2082 0 -1 4202560 877 0 0 0 0 1 0 0 18 0 1 0 657900885 287498240 6001 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709135 0 0 16781312 514 15566631368283619589 0 0 17 1 0 0 0
/proc/2232/statm: 70190 6001 378 364 0 6108 0
[pid=2234] ppid=2232 vsize=1019708 CPUtime=4.09 cores=1,3,5,7
/proc/2234/stat : 2234 (gnovelty+2) R 2232 2234 2234 0 -1 4202496 950 0 0 0 409 0 0 0 25 0 1 0 657900885 1044180992 690 33554432000 134512640 135663704 4294747536 18446744073709551615 134525499 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/2234/statm: 254927 690 113 282 0 254643 0
[pid=2235] ppid=2226 vsize=291084 CPUtime=0.01 cores=1,3,5,7
/proc/2235/stat : 2235 (python) S 2226 2226 2082 0 -1 4202560 895 0 0 0 0 1 0 0 18 0 1 0 657900886 298070016 5999 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 4456184292229218564 0 0 17 7 0 0 0
/proc/2235/statm: 72771 5999 372 364 0 8686 0
[pid=2237] ppid=2235 vsize=1019708 CPUtime=4.1 cores=1,3,5,7
/proc/2237/stat : 2237 (gnovelty+2) R 2235 2237 2237 0 -1 4202496 951 0 0 0 410 0 0 0 25 0 1 0 657900886 1044180992 690 33554432000 134512640 135663704 4294517120 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/2237/statm: 254927 690 113 282 0 254643 0
[pid=2238] ppid=2226 vsize=301272 CPUtime=0.01 cores=1,3,5,7
/proc/2238/stat : 2238 (python) S 2226 2226 2082 0 -1 4202560 897 0 0 0 0 1 0 0 18 0 1 0 657900887 308502528 6020 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 15566623671702225156 0 0 17 5 0 0 0
/proc/2238/statm: 75318 6020 372 364 0 11230 0
[pid=2240] ppid=2238 vsize=420220 CPUtime=4.01 cores=1,3,5,7
/proc/2240/stat : 2240 (adaptg2wsat2009) R 2238 2240 2240 0 -1 4202496 1666 0 0 0 401 0 0 0 25 0 1 0 657900887 430305280 1399 33554432000 134512640 135107411 4288367632 18446744073709551615 134531797 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/2240/statm: 105055 1399 59 146 0 103931 0
[pid=2241] ppid=2226 vsize=311596 CPUtime=0.01 cores=1,3,5,7
/proc/2241/stat : 2241 (python) S 2226 2226 2082 0 -1 4202560 907 0 0 0 0 1 0 0 18 0 1 0 657900888 319074304 6024 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 12702975323973583111 0 0 17 5 0 0 0
/proc/2241/statm: 77899 6024 372 364 0 13808 0
[pid=2252] ppid=2241 vsize=420220 CPUtime=4.04 cores=1,3,5,7
/proc/2252/stat : 2252 (adaptg2wsat2009) R 2241 2252 2252 0 -1 4202496 1659 0 0 0 404 0 0 0 25 0 1 0 657900888 430305280 1400 33554432000 134512640 135107411 4290163440 18446744073709551615 134532293 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/2252/statm: 105055 1400 59 146 0 103931 0
[pid=2226/tid=2233] ppid=2224 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2233/stat : 2233 (python) S 2224 2226 2082 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 5 0 657900885 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
[pid=2226/tid=2236] ppid=2224 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2236/stat : 2236 (python) S 2224 2226 2082 0 -1 4202560 22 932 0 0 0 0 0 0 21 0 5 0 657900886 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
[pid=2226/tid=2239] ppid=2224 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2239/stat : 2239 (python) S 2224 2226 2082 0 -1 4202560 23 932 0 0 0 0 0 0 23 0 5 0 657900887 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
[pid=2226/tid=2245] ppid=2224 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2245/stat : 2245 (python) S 2224 2226 2082 0 -1 4202560 29 932 0 0 0 0 0 0 25 0 5 0 657900888 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 18.24
Current children cumulated vsize (KiB) 4385856

[startup+12.701 s]
/proc/loadavg: 6.11 3.93 3.80 9/388 2252
/proc/meminfo: memFree=26636452/32950928 swapFree=60199044/67111528
[pid=2226] ppid=2224 vsize=321288 CPUtime=1.96 cores=1,3,5,7
/proc/2226/stat : 2226 (python) S 2224 2226 2082 0 -1 4202496 15256 932 1 0 183 13 0 0 25 0 5 0 657900667 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/2226/statm: 80322 7530 1875 364 0 16231 0
[pid=2232] ppid=2226 vsize=280760 CPUtime=0.03 cores=1,3,5,7
/proc/2232/stat : 2232 (python) S 2226 2226 2082 0 -1 4202560 880 0 0 0 1 2 0 0 18 0 1 0 657900885 287498240 6001 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709135 0 0 16781312 514 7241876161741947141 0 0 17 1 0 0 0
/proc/2232/statm: 70190 6001 378 364 0 6108 0
[pid=2234] ppid=2232 vsize=1019708 CPUtime=10.47 cores=1,3,5,7
/proc/2234/stat : 2234 (gnovelty+2) R 2232 2234 2234 0 -1 4202496 950 0 0 0 1047 0 0 0 25 0 1 0 657900885 1044180992 690 33554432000 134512640 135663704 4294747536 18446744073709551615 134525224 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/2234/statm: 254927 690 113 282 0 254643 0
[pid=2235] ppid=2226 vsize=291084 CPUtime=0.03 cores=1,3,5,7
/proc/2235/stat : 2235 (python) S 2226 2226 2082 0 -1 4202560 902 0 0 0 1 2 0 0 18 0 1 0 657900886 298070016 5999 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 7241876161741947140 0 0 17 7 0 0 0
/proc/2235/statm: 72771 5999 372 364 0 8686 0
[pid=2237] ppid=2235 vsize=1019708 CPUtime=10.5 cores=1,3,5,7
/proc/2237/stat : 2237 (gnovelty+2) R 2235 2237 2237 0 -1 4202496 951 0 0 0 1050 0 0 0 25 0 1 0 657900886 1044180992 690 33554432000 134512640 135663704 4294517120 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/2237/statm: 254927 690 113 282 0 254643 0
[pid=2238] ppid=2226 vsize=301272 CPUtime=0.03 cores=1,3,5,7
/proc/2238/stat : 2238 (python) S 2226 2226 2082 0 -1 4202560 900 0 0 0 1 2 0 0 18 0 1 0 657900887 308502528 6020 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 7241876161741947140 0 0 17 5 0 0 0
/proc/2238/statm: 75318 6020 372 364 0 11230 0
[pid=2240] ppid=2238 vsize=420220 CPUtime=10.32 cores=1,3,5,7
/proc/2240/stat : 2240 (adaptg2wsat2009) R 2238 2240 2240 0 -1 4202496 1666 0 0 0 1032 0 0 0 25 0 1 0 657900887 430305280 1399 33554432000 134512640 135107411 4288367632 18446744073709551615 134531600 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/2240/statm: 105055 1399 59 146 0 103931 0
[pid=2241] ppid=2226 vsize=311596 CPUtime=0.03 cores=1,3,5,7
/proc/2241/stat : 2241 (python) S 2226 2226 2082 0 -1 4202560 912 0 0 0 1 2 0 0 18 0 1 0 657900888 319074304 6024 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 7241876161741947143 0 0 17 5 0 0 0
/proc/2241/statm: 77899 6024 372 364 0 13808 0
[pid=2252] ppid=2241 vsize=420220 CPUtime=10.41 cores=1,3,5,7
/proc/2252/stat : 2252 (adaptg2wsat2009) R 2241 2252 2252 0 -1 4202496 1659 0 0 0 1041 0 0 0 25 0 1 0 657900888 430305280 1400 33554432000 134512640 135107411 4290163440 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/2252/statm: 105055 1400 59 146 0 103931 0
[pid=2226/tid=2233] ppid=2224 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2233/stat : 2233 (python) S 2224 2226 2082 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 5 0 657900885 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
[pid=2226/tid=2236] ppid=2224 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2236/stat : 2236 (python) S 2224 2226 2082 0 -1 4202560 22 932 0 0 0 0 0 0 21 0 5 0 657900886 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
[pid=2226/tid=2239] ppid=2224 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2239/stat : 2239 (python) S 2224 2226 2082 0 -1 4202560 23 932 0 0 0 0 0 0 23 0 5 0 657900887 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
[pid=2226/tid=2245] ppid=2224 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2245/stat : 2245 (python) S 2224 2226 2082 0 -1 4202560 29 932 0 0 0 0 0 0 25 0 5 0 657900888 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 43.78
Current children cumulated vsize (KiB) 4385856

[startup+25.5007 s]
/proc/loadavg: 6.43 4.11 3.86 6/376 2284
/proc/meminfo: memFree=26792620/32950928 swapFree=60199044/67111528
[pid=2226] ppid=2224 vsize=321288 CPUtime=1.96 cores=1,3,5,7
/proc/2226/stat : 2226 (python) S 2224 2226 2082 0 -1 4202496 15256 932 1 0 183 13 0 0 25 0 5 0 657900667 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/2226/statm: 80322 7530 1875 364 0 16231 0
[pid=2232] ppid=2226 vsize=280760 CPUtime=0.06 cores=1,3,5,7
/proc/2232/stat : 2232 (python) S 2226 2226 2082 0 -1 4202560 886 0 0 0 3 3 0 0 18 0 1 0 657900885 287498240 6001 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709135 0 0 16781312 514 7241876161741947141 0 0 17 1 0 0 0
/proc/2232/statm: 70190 6001 378 364 0 6108 0
[pid=2234] ppid=2232 vsize=1019708 CPUtime=23.24 cores=1,3,5,7
/proc/2234/stat : 2234 (gnovelty+2) R 2232 2234 2234 0 -1 4202496 950 0 0 0 2324 0 0 0 25 0 1 0 657900885 1044180992 690 33554432000 134512640 135663704 4294747536 18446744073709551615 134544380 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/2234/statm: 254927 690 113 282 0 254643 0
[pid=2235] ppid=2226 vsize=291084 CPUtime=0.06 cores=1,3,5,7
/proc/2235/stat : 2235 (python) S 2226 2226 2082 0 -1 4202560 908 0 0 0 3 3 0 0 18 0 1 0 657900886 298070016 5999 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 7241876161741947140 0 0 17 7 0 0 0
/proc/2235/statm: 72771 5999 372 364 0 8686 0
[pid=2237] ppid=2235 vsize=1019708 CPUtime=23.3 cores=1,3,5,7
/proc/2237/stat : 2237 (gnovelty+2) R 2235 2237 2237 0 -1 4202496 951 0 0 0 2330 0 0 0 25 0 1 0 657900886 1044180992 690 33554432000 134512640 135663704 4294517120 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/2237/statm: 254927 690 113 282 0 254643 0
[pid=2238] ppid=2226 vsize=301272 CPUtime=0.06 cores=1,3,5,7
/proc/2238/stat : 2238 (python) S 2226 2226 2082 0 -1 4202560 904 0 0 0 2 4 0 0 18 0 1 0 657900887 308502528 6020 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 7241876161741947140 0 0 17 5 0 0 0
/proc/2238/statm: 75318 6020 372 364 0 11230 0
[pid=2240] ppid=2238 vsize=420220 CPUtime=22.93 cores=1,3,5,7
/proc/2240/stat : 2240 (adaptg2wsat2009) R 2238 2240 2240 0 -1 4202496 1666 0 0 0 2293 0 0 0 25 0 1 0 657900887 430305280 1399 33554432000 134512640 135107411 4288367632 18446744073709551615 134532224 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/2240/statm: 105055 1399 59 146 0 103931 0
[pid=2241] ppid=2226 vsize=311596 CPUtime=0.06 cores=1,3,5,7
/proc/2241/stat : 2241 (python) S 2226 2226 2082 0 -1 4202560 916 0 0 0 2 4 0 0 18 0 1 0 657900888 319074304 6024 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 7241876161741947143 0 0 17 5 0 0 0
/proc/2241/statm: 77899 6024 372 364 0 13808 0
[pid=2252] ppid=2241 vsize=420220 CPUtime=23.14 cores=1,3,5,7
/proc/2252/stat : 2252 (adaptg2wsat2009) R 2241 2252 2252 0 -1 4202496 1659 0 0 0 2314 0 0 0 25 0 1 0 657900888 430305280 1400 33554432000 134512640 135107411 4290163440 18446744073709551615 134532107 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/2252/statm: 105055 1400 59 146 0 103931 0
[pid=2226/tid=2233] ppid=2224 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2233/stat : 2233 (python) S 2224 2226 2082 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 5 0 657900885 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
[pid=2226/tid=2236] ppid=2224 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2236/stat : 2236 (python) S 2224 2226 2082 0 -1 4202560 22 932 0 0 0 0 0 0 21 0 5 0 657900886 328998912 7530 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
[pid=2226/tid=2239] ppid=2224 vsize=321288 CPUtime=0 cores=1,3,5,7

################
# More data... #
################

/proc/2238/statm: 75318 6049 400 364 0 11230 0
[pid=2240] ppid=2238 vsize=420220 CPUtime=98.76 cores=1,3,5,7
/proc/2240/stat : 2240 (adaptg2wsat2009) T 2238 2240 2240 0 -1 4202496 1666 0 0 0 9876 0 0 0 25 0 1 0 657900887 430305280 1399 33554432000 134512640 135107411 4288367632 18446744073709551615 134531435 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/2240/statm: 105055 1399 59 146 0 103931 0
[pid=2238/tid=2306] ppid=2226 vsize=301272 CPUtime=0 cores=1,3,5,7
/proc/2238/task/2306/stat : 2306 (python) S 2226 2226 2082 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 657910915 308502528 6049 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604460832047552 0 0 -1 7 0 0 0
[pid=2241] ppid=2226 vsize=311596 CPUtime=0.26 cores=1,3,5,7
/proc/2241/stat : 2241 (python) S 2226 2226 2082 0 -1 4202560 1090 0 0 0 10 16 0 0 19 0 2 0 657900888 319074304 6053 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/2241/statm: 77899 6053 400 364 0 13808 0
[pid=2252] ppid=2241 vsize=420220 CPUtime=99.75 cores=1,3,5,7
/proc/2252/stat : 2252 (adaptg2wsat2009) T 2241 2252 2252 0 -1 4202496 1659 0 0 0 9975 0 0 0 25 0 1 0 657900888 430305280 1400 33554432000 134512640 135107411 4290163440 18446744073709551615 134531822 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/2252/statm: 105055 1400 59 146 0 103931 0
[pid=2241/tid=2310] ppid=2226 vsize=311596 CPUtime=0 cores=1,3,5,7
/proc/2241/task/2310/stat : 2310 (python) S 2226 2226 2082 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657910917 319074304 6053 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604460165302464 0 0 -1 5 0 0 0
[pid=2303] ppid=2226 vsize=321928 CPUtime=0.28 cores=1,3,5,7
/proc/2303/stat : 2303 (python) S 2226 2226 2082 0 -1 4202560 1075 0 0 0 15 13 0 0 19 0 2 0 657910722 329654272 6057 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/2303/statm: 80482 6057 400 364 0 16388 0
[pid=2305] ppid=2303 vsize=420220 CPUtime=99.65 cores=1,3,5,7
/proc/2305/stat : 2305 (adaptg2wsat2009) T 2303 2305 2305 0 -1 4202496 1660 0 0 0 9965 0 0 0 25 0 1 0 657910722 430305280 1399 33554432000 134512640 135107411 4291661392 18446744073709551615 134531435 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/2305/statm: 105055 1399 59 146 0 103931 0
[pid=2303/tid=2412] ppid=2226 vsize=321928 CPUtime=0 cores=1,3,5,7
/proc/2303/task/2412/stat : 2412 (python) S 2226 2226 2082 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657920749 329654272 6057 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604464029480128 0 0 -1 1 0 0 0
[pid=2307] ppid=2226 vsize=332264 CPUtime=0.29 cores=1,3,5,7
/proc/2307/stat : 2307 (python) S 2226 2226 2082 0 -1 4202560 1098 0 0 0 15 14 0 0 19 0 2 0 657910916 340238336 6080 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/2307/statm: 83066 6080 400 364 0 18969 0
[pid=2311] ppid=2307 vsize=1019708 CPUtime=100.29 cores=1,3,5,7
/proc/2311/stat : 2311 (gnovelty+2) T 2307 2311 2311 0 -1 4202496 947 0 0 0 10029 0 0 0 25 0 1 0 657910918 1044180992 689 33554432000 134512640 135663704 4290101968 18446744073709551615 134544240 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/2311/statm: 254927 689 113 282 0 254643 0
[pid=2307/tid=2424] ppid=2226 vsize=332264 CPUtime=0 cores=1,3,5,7
/proc/2307/task/2424/stat : 2424 (python) S 2226 2226 2082 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657920949 340238336 6080 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604464029481920 0 0 -1 5 0 0 0
[pid=2312] ppid=2226 vsize=342468 CPUtime=0.28 cores=1,3,5,7
/proc/2312/stat : 2312 (python) S 2226 2226 2082 0 -1 4202560 1096 0 0 0 14 14 0 0 19 0 2 0 657910918 350687232 6104 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/2312/statm: 85617 6104 400 364 0 21517 0
[pid=2314] ppid=2312 vsize=249872 CPUtime=99.73 cores=1,3,5,7
/proc/2314/stat : 2314 (FH) T 2312 2314 2314 0 -1 4202496 1527 0 0 0 9973 0 0 0 25 0 1 0 657910918 255868928 1263 33554432000 134512640 134936988 4294551872 18446744073709551615 134553505 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/2314/statm: 62468 1263 41 104 0 61972 0
[pid=2312/tid=2416] ppid=2226 vsize=342468 CPUtime=0 cores=1,3,5,7
/proc/2312/task/2416/stat : 2416 (python) S 2226 2226 2082 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 657920946 350687232 6104 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604470157249216 0 0 -1 1 0 0 0
[pid=2315] ppid=2226 vsize=352824 CPUtime=0.28 cores=1,3,5,7
/proc/2315/stat : 2315 (python) S 2226 2226 2082 0 -1 4202560 1132 0 0 0 14 14 0 0 19 0 2 0 657910919 361291776 6114 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/2315/statm: 88206 6114 406 364 0 24103 0
[pid=2317] ppid=2315 vsize=420220 CPUtime=98.11 cores=1,3,5,7
/proc/2317/stat : 2317 (adaptg2wsat2009) T 2315 2317 2317 0 -1 4202496 1670 0 0 0 9811 0 0 0 25 0 1 0 657910919 430305280 1400 33554432000 134512640 135107411 4291183760 18446744073709551615 134532128 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/2317/statm: 105055 1400 59 146 0 103931 0
[pid=2315/tid=2417] ppid=2226 vsize=352824 CPUtime=0 cores=1,3,5,7
/proc/2315/task/2417/stat : 2417 (python) S 2226 2226 2082 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657920947 361291776 6114 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604454223471552 0 0 -1 5 0 0 0
[pid=2413] ppid=2226 vsize=363184 CPUtime=0.29 cores=1,3,5,7
/proc/2413/stat : 2413 (python) S 2226 2226 2082 0 -1 4202560 1148 0 0 0 14 15 0 0 19 0 2 0 657920751 371900416 6130 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/2413/statm: 90796 6130 406 364 0 26690 0
[pid=2415] ppid=2413 vsize=420220 CPUtime=97.49 cores=1,3,5,7
/proc/2415/stat : 2415 (adaptg2wsat2009) T 2413 2415 2415 0 -1 4202496 1671 0 0 0 9749 0 0 0 25 0 1 0 657920751 430305280 1399 33554432000 134512640 135107411 4290350352 18446744073709551615 134531781 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/2415/statm: 105055 1399 59 146 0 103931 0
[pid=2413/tid=2447] ppid=2226 vsize=363184 CPUtime=0 cores=1,3,5,7
/proc/2413/task/2447/stat : 2447 (python) S 2226 2226 2082 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657930779 371900416 6130 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604470157248320 0 0 -1 5 0 0 0
[pid=2418] ppid=2226 vsize=373552 CPUtime=0.28 cores=1,3,5,7
/proc/2418/stat : 2418 (python) S 2226 2226 2082 0 -1 4202560 1105 0 0 0 15 13 0 0 19 0 2 0 657920947 382517248 6154 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/2418/statm: 93388 6154 400 364 0 29279 0
[pid=2420] ppid=2418 vsize=249876 CPUtime=100.27 cores=1,3,5,7
/proc/2420/stat : 2420 (FH) T 2418 2420 2420 0 -1 4202496 1525 0 0 0 10027 0 0 0 25 0 1 0 657920947 255873024 1263 33554432000 134512640 134936988 4294510848 18446744073709551615 134554983 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/2420/statm: 62469 1263 41 104 0 61973 0
[pid=2418/tid=2455] ppid=2226 vsize=373552 CPUtime=0 cores=1,3,5,7
/proc/2418/task/2455/stat : 2455 (python) S 2226 2226 2082 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657930975 382517248 6154 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604468994026176 0 0 -1 1 0 0 0
[pid=2421] ppid=2226 vsize=383812 CPUtime=0.9 cores=1,3,5,7
/proc/2421/stat : 2421 (python) S 2226 2226 2082 0 -1 4202560 1161 0 0 0 45 45 0 0 18 0 2 0 657920948 393023488 6186 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/2421/statm: 95953 6186 400 364 0 31841 0
[pid=2423] ppid=2421 vsize=268424 CPUtime=292.56 cores=1,3,5,7
/proc/2423/stat : 2423 (hybridGM3) T 2421 2423 2423 0 -1 4202496 5389 0 0 0 29256 0 0 0 25 0 1 0 657920949 274866176 5126 33554432000 134512640 135204385 4292746112 18446744073709551615 134538282 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/2423/statm: 67106 5126 63 169 0 66936 0
[pid=2421/tid=2468] ppid=2226 vsize=383812 CPUtime=0 cores=1,3,5,7
/proc/2421/task/2468/stat : 2468 (python) S 2226 2226 2082 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 657940416 393023488 6186 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604457396923264 0 0 -1 5 0 0 0
[pid=2425] ppid=2226 vsize=394196 CPUtime=0.29 cores=1,3,5,7
/proc/2425/stat : 2425 (python) S 2226 2226 2082 0 -1 4202560 1135 0 0 0 15 14 0 0 19 0 2 0 657920950 403656704 6191 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/2425/statm: 98549 6191 400 364 0 34434 0
[pid=2427] ppid=2425 vsize=1019708 CPUtime=99.05 cores=1,3,5,7
/proc/2427/stat : 2427 (gnovelty+2) T 2425 2427 2427 0 -1 4202496 958 0 0 0 9905 0 0 0 25 0 1 0 657920951 1044180992 690 33554432000 134512640 135663704 4293051536 18446744073709551615 134525278 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/2427/statm: 254927 690 113 282 0 254643 0
[pid=2425/tid=2451] ppid=2226 vsize=394196 CPUtime=0 cores=1,3,5,7
/proc/2425/task/2451/stat : 2451 (python) S 2226 2226 2082 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657930884 403656704 6191 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604457396921472 0 0 -1 1 0 0 0
[pid=2448] ppid=2226 vsize=404240 CPUtime=0.59 cores=1,3,5,7
/proc/2448/stat : 2448 (python) S 2226 2226 2082 0 -1 4202560 1005 0 0 0 32 27 0 0 18 0 1 0 657930780 413941760 6198 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 7241876161741947143 0 0 17 7 0 0 0
/proc/2448/statm: 101060 6198 372 364 0 36942 0
[pid=2450] ppid=2448 vsize=268424 CPUtime=194.69 cores=1,3,5,7
/proc/2450/stat : 2450 (hybridGM3) R 2448 2450 2450 0 -1 4202496 5397 0 0 0 19469 0 0 0 25 0 1 0 657930780 274866176 5127 33554432000 134512640 135204385 4286702432 18446744073709551615 134545446 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/2450/statm: 67106 5127 63 169 0 66936 0
[pid=2452] ppid=2226 vsize=414640 CPUtime=0.3 cores=1,3,5,7
/proc/2452/stat : 2452 (python) S 2226 2226 2082 0 -1 4202560 1150 0 0 0 16 14 0 0 19 0 2 0 657930885 424591360 6231 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/2452/statm: 103660 6231 400 364 0 39539 0
[pid=2454] ppid=2452 vsize=420220 CPUtime=99.27 cores=1,3,5,7
/proc/2454/stat : 2454 (adaptg2wsat2009) T 2452 2454 2454 0 -1 4202496 1670 0 0 0 9927 0 0 0 25 0 1 0 657930887 430305280 1400 33554432000 134512640 135107411 4290351024 18446744073709551615 134532313 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/2454/statm: 105055 1400 59 146 0 103931 0
[pid=2452/tid=2470] ppid=2226 vsize=414640 CPUtime=0 cores=1,3,5,7
/proc/2452/task/2470/stat : 2470 (python) S 2226 2226 2082 0 -1 4202560 52 0 0 0 0 0 0 0 19 0 2 0 657940817 424591360 6231 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604470214800640 0 0 -1 7 0 0 0
[pid=2456] ppid=2226 vsize=425044 CPUtime=0.6 cores=1,3,5,7
/proc/2456/stat : 2456 (python) S 2226 2226 2082 0 -1 4202560 1230 0 0 0 31 29 0 0 19 0 2 0 657930977 435245056 6243 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/2456/statm: 106261 6243 408 364 0 42137 0
[pid=2458] ppid=2456 vsize=268420 CPUtime=195.67 cores=1,3,5,7
/proc/2458/stat : 2458 (hybridGM3) T 2456 2458 2458 0 -1 4202496 5400 0 0 0 19567 0 0 0 25 0 1 0 657930977 274862080 5126 33554432000 134512640 135204385 4289854432 18446744073709551615 134538475 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/2458/statm: 67105 5126 63 169 0 66935 0
[pid=2456/tid=2492] ppid=2226 vsize=425044 CPUtime=0 cores=1,3,5,7
/proc/2456/task/2492/stat : 2492 (python) S 2226 2226 2082 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 657950545 435245056 6243 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 514 18446604471136364608 0 0 -1 3 0 0 0
[pid=2471] ppid=2226 vsize=435084 CPUtime=0.29 cores=1,3,5,7
/proc/2471/stat : 2471 (python) S 2226 2226 2082 0 -1 4202560 1035 0 0 0 15 14 0 0 18 0 1 0 657940819 445526016 6238 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 7241876161741947144 0 0 17 7 0 0 0
/proc/2471/statm: 108771 6238 380 364 0 44644 0
[pid=2473] ppid=2471 vsize=268424 CPUtime=93.55 cores=1,3,5,7
/proc/2473/stat : 2473 (hybridGM3) R 2471 2473 2473 0 -1 4202496 5401 0 0 0 9355 0 0 0 25 0 1 0 657940819 274866176 5126 33554432000 134512640 135204385 4291297296 18446744073709551615 134538480 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/2473/statm: 67106 5126 63 169 0 66936 0
[pid=2489] ppid=2226 vsize=445760 CPUtime=0 cores=1,3,5,7
/proc/2489/stat : 2489 (python) S 2226 2226 2082 0 -1 4202560 850 0 0 0 0 0 0 0 18 0 1 0 657950252 456458240 6219 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 10317008703420596485 0 0 17 7 0 0 0
/proc/2489/statm: 111440 6219 357 364 0 47310 0
[pid=2491] ppid=2489 vsize=454088 CPUtime=3.32 cores=1,3,5,7
/proc/2491/stat : 2491 (TNM) R 2489 2491 2491 0 -1 4202496 1274 0 0 0 332 0 0 0 25 0 1 0 657950252 464986112 995 33554432000 134512640 135008748 4289203072 18446744073709551615 134523507 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/2491/statm: 113522 995 45 122 0 112422 0
[pid=2493] ppid=2226 vsize=456188 CPUtime=0 cores=1,3,5,7
/proc/2493/stat : 2493 (python) S 2226 2226 2082 0 -1 4202560 824 0 0 0 0 0 0 0 19 0 1 0 657950547 467136512 6213 33554432000 4194304 5683644 140735438242800 18446744073709551615 268272709190 0 0 16781312 514 10317012551711326207 0 0 17 7 0 0 0
/proc/2493/statm: 114047 6213 347 364 0 49914 0
[pid=2495] ppid=2493 vsize=247956 CPUtime=0.51 cores=1,3,5,7
/proc/2495/stat : 2495 (FH) R 2493 2495 2495 0 -1 4202496 1262 0 0 0 51 0 0 0 22 0 1 0 657950547 253906944 987 33554432000 134512640 134936988 4292163024 18446744073709551615 134523622 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/2495/statm: 61989 987 37 104 0 61493 0
[pid=2226/tid=2233] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2233/stat : 2233 (python) S 2224 2226 2082 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 19 0 657900885 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
[pid=2226/tid=2236] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2236/stat : 2236 (python) S 2224 2226 2082 0 -1 4202560 22 932 0 0 0 0 0 0 21 0 19 0 657900886 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
[pid=2226/tid=2239] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2239/stat : 2239 (python) S 2224 2226 2082 0 -1 4202560 23 932 0 0 0 0 0 0 23 0 19 0 657900887 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
[pid=2226/tid=2245] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2245/stat : 2245 (python) S 2224 2226 2082 0 -1 4202560 29 932 0 0 0 0 0 0 25 0 19 0 657900888 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
[pid=2226/tid=2304] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2304/stat : 2304 (python) S 2224 2226 2082 0 -1 4202560 28 932 0 0 0 0 0 0 20 0 19 0 657910722 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
[pid=2226/tid=2308] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2308/stat : 2308 (python) S 2224 2226 2082 0 -1 4202560 24 932 0 0 0 0 0 0 20 0 19 0 657910916 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 3 0 0 0
[pid=2226/tid=2313] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2313/stat : 2313 (python) S 2224 2226 2082 0 -1 4202560 24 932 0 0 0 0 0 0 21 0 19 0 657910918 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 1 0 0 0
[pid=2226/tid=2316] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2316/stat : 2316 (python) S 2224 2226 2082 0 -1 4202560 26 932 0 0 0 0 0 0 23 0 19 0 657910919 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
[pid=2226/tid=2414] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2414/stat : 2414 (python) S 2224 2226 2082 0 -1 4202560 24 932 0 0 0 0 0 0 20 0 19 0 657920751 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 5 0 0 0
[pid=2226/tid=2419] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2419/stat : 2419 (python) S 2224 2226 2082 0 -1 4202560 23 932 0 0 0 0 0 0 20 0 19 0 657920947 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 1 0 0 0
[pid=2226/tid=2422] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2422/stat : 2422 (python) S 2224 2226 2082 0 -1 4202560 31 932 0 0 0 0 0 0 18 0 19 0 657920949 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
[pid=2226/tid=2426] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2426/stat : 2426 (python) S 2224 2226 2082 0 -1 4202560 26 932 0 0 0 0 0 0 23 0 19 0 657920951 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
[pid=2226/tid=2449] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2449/stat : 2449 (python) S 2224 2226 2082 0 -1 4202560 25 932 0 0 0 0 0 0 20 0 19 0 657930780 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
[pid=2226/tid=2453] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2453/stat : 2453 (python) S 2224 2226 2082 0 -1 4202560 27 932 0 0 0 0 0 0 20 0 19 0 657930885 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
[pid=2226/tid=2457] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2457/stat : 2457 (python) S 2224 2226 2082 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 19 0 657930977 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
[pid=2226/tid=2472] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2472/stat : 2472 (python) S 2224 2226 2082 0 -1 4202560 29 932 0 0 0 0 0 0 20 0 19 0 657940819 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
[pid=2226/tid=2490] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2490/stat : 2490 (python) S 2224 2226 2082 0 -1 4202560 28 932 0 0 0 0 0 0 20 0 19 0 657950252 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
[pid=2226/tid=2494] ppid=2224 vsize=465492 CPUtime=0 cores=1,3,5,7
/proc/2226/task/2494/stat : 2494 (python) S 2224 2226 2082 0 -1 4202560 28 932 0 0 0 0 0 0 20 0 19 0 657950547 476663808 7781 33554432000 4194304 5683644 140735438242800 18446744073709551615 268284512689 0 0 16781312 2 18446604471079705536 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1978.95
Current children cumulated vsize (KiB) 15981268

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 2226 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=150673
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=325950
# CPU time returned by wait4() is 2.47662
# while last known CPU time is 1978.95
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 499.455
CPU time (s): 1978.95
CPU user time (s): 1975.91
CPU system time (s): 3.04
CPU usage (%): 396.222
Max. virtual memory (cumulated for all children) (KiB): 15981268

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.15067
system time used= 0.32595
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 30361
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3426
involuntary context switches= 73

runsolver used 3.37049 second user time and 7.81481 second system time

The end

Launcher Data 

Begin job on node122 at 2011-05-15 14:36:59
IDJOB=3378133
IDBENCH=82542
IDSOLVER=1859
FILE ID=node122/3378133-1305463018
RUNJOBID= node122-1305462946-2101
PBS_JOBID= 13324977
Free space on /tmp= 73424 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v300-c25500-S942666749-061-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378133-1305463018/watcher-3378133-1305463018 -o /tmp/evaluation-result-3378133-1305463018/solver-3378133-1305463018 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378133-1305463018.cnf -seed 570989988 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 456ae67a44e812130272597099547869
RANDOM SEED=570989988

node122.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      26695084 kB
Buffers:        308764 kB
Cached:        3920088 kB
SwapCached:      50104 kB
Active:        1899944 kB
Inactive:      4119044 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      26695084 kB
SwapTotal:    67111528 kB
SwapFree:     60199044 kB
Dirty:           32220 kB
Writeback:           0 kB
AnonPages:     1750932 kB
Mapped:          24964 kB
Slab:           129528 kB
PageTables:      41740 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 81111420 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73396 MiB
End job on node122 at 2011-05-15 14:45:20