Trace number 3378134

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 1985.52 501.635

General information on the benchmark

Namerandom/large/
unif-k5-r20-v2000-c40000-S319509287-085-UNKNOWN.cnf
MD5SUMbb548a1e884feefbb349d3449e849f5e
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2000
Number of clauses40000
Sum of the clauses size200000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 540000
Number of clauses of size over 50

Solver Data 

0.39/0.79	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.39/0.79	c INFO: solving HOME/instance-3378134-1305463021.cnf
0.39/0.79	c INFO: computing task features
0.56/1.01	c INFO: computed variable-clause graph statistics
0.67/1.01	c INFO: computed clause balance statistics
1.56/1.98	c INFO: computed variable balance statistics
2.16/2.53	c INFO: computed small-clause counts
2.16/2.53	c INFO: computed Horn-clause counts
2.16/2.58	c INFO: computed variable graph statistics
2.16/2.58	c INFO: collected features for HOME/instance-3378134-1305463021.cnf in 1.77 s
2.16/2.58	c INFO: running gnovelty+2@0 for 97 with 1198 remaining (b = 0.38)
2.25/2.60	c INFO: running gnovelty+2@0 for 97 with 1198 remaining (b = 0.13)
2.25/2.61	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.08)
2.25/2.62	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.07)
401.92/102.92	c INFO: running adaptg2wsat2009++@0 for 97 with 1097 remaining (b = 0.05)
401.92/102.95	c INFO: running gnovelty+2@0 for 97 with 1097 remaining (b = 0.05)
401.92/102.97	c INFO: running FH@0 for 97 with 1097 remaining (b = 0.03)
402.15/103.04	c INFO: running adaptg2wsat2009++@0 for 97 with 1097 remaining (b = 0.03)
800.61/203.23	c INFO: running adaptg2wsat2009++@0 for 97 with 997 remaining (b = 0.02)
800.61/203.29	c INFO: running FH@0 for 97 with 997 remaining (b = 0.02)
800.94/203.36	c INFO: running hybridGM3@0 for 194 with 997 remaining (b = 0.04)
800.94/203.38	c INFO: running gnovelty+2@0 for 97 with 997 remaining (b = 0.02)
1199.35/303.55	c INFO: running hybridGM3@0 for 194 with 897 remaining (b = 0.03)
1199.35/303.61	c INFO: running adaptg2wsat2009++@0 for 97 with 897 remaining (b = 0.01)
1199.85/303.75	c INFO: running hybridGM3@0 for 194 with 897 remaining (b = 0.03)
1579.19/399.17	c INFO: running hybridGM3@100 for 97 with 801 remaining (b = 0.01)
1598.24/403.94	c INFO: running hybridGM3@0 for 194 with 796 remaining (b = 0.02)
1980.31/500.26	c INFO: running hybridGM3@100 for 97 with 700 remaining (b = 0.01)
1981.14/500.45	c INFO: running TNM@0 for 700 with 700 remaining (b = 0.08)
1981.14/500.51	c INFO: running FH@0 for 97 with 700 remaining (b = 0.01)
1985.52/501.62	c INFO: running adaptg2wsat2009++@99 for 97 with 699 remaining (b = 0.01)
1985.52/501.62	Traceback (most recent call last):
1985.52/501.62	  File "HOME/solve", line 33, in <module>
1985.52/501.62	    plac.call(main)
1985.52/501.62	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call
1985.52/501.63	    cmd, result = parser_from(obj).consume(arglist)
1985.52/501.63	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 192, in consume
1985.52/501.63	    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
1985.52/501.63	  File "HOME/borg/build/lib/borg/tools/solve.py", line 84, in main
1985.52/501.63	    (_, answer, _) = solver(input_path, budget / cores, cores)
1985.52/501.63	  File "HOME/borg/build/lib/borg/bilevel.py", line 202, in __call__
1985.52/501.63	    solver.go(max_cost * borg.defaults.machine_speed)
1985.52/501.63	  File "HOME/borg/build/lib/borg/solvers.py", line 179, in go
1985.52/501.63	    self._process.start()
1985.52/501.63	  File "/home/evaluation/evaluation/tools64/lib/python2.7/multiprocessing/process.py", line 94, in start
1985.52/501.63	    assert self._popen is None, 'cannot start a process twice'
1985.52/501.63	AssertionError: cannot start a process twice

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378134-1305463021/watcher-3378134-1305463021 -o /tmp/evaluation-result-3378134-1305463021/solver-3378134-1305463021 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378134-1305463021.cnf -seed 1937025153 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.31 2.98 3.55 2/386 30746
/proc/meminfo: memFree=21974964/32951124 swapFree=67111348/67111528
[pid=30746] ppid=30743 vsize=15772 CPUtime=0 cores=0,2,4,6
/proc/30746/stat : 30746 (python) S 30743 30746 30709 0 -1 4202496 925 0 0 0 0 0 0 0 17 0 1 0 657899858 16150528 806 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687673344 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/30746/statm: 3943 806 356 364 0 430 0

[startup+0.0831829 s]
/proc/loadavg: 1.31 2.98 3.55 2/386 30746
/proc/meminfo: memFree=21974964/32951124 swapFree=67111348/67111528
[pid=30746] ppid=30743 vsize=36504 CPUtime=0.04 cores=0,2,4,6
/proc/30746/stat : 30746 (python) S 30743 30746 30709 0 -1 4202496 1608 0 0 0 3 1 0 0 17 0 1 0 657899858 37380096 1443 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687673344 0 2147483391 16781312 2 18446744071699945619 0 0 17 4 0 0 0
/proc/30746/statm: 9126 1443 452 364 0 943 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 36504

[startup+0.101183 s]
/proc/loadavg: 1.31 2.98 3.55 2/386 30746
/proc/meminfo: memFree=21974964/32951124 swapFree=67111348/67111528
[pid=30746] ppid=30743 vsize=50008 CPUtime=0.06 cores=0,2,4,6
/proc/30746/stat : 30746 (python) S 30743 30746 30709 0 -1 4202496 1973 0 0 0 4 2 0 0 17 0 1 0 657899858 51208192 1736 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687673344 0 2147483391 16781312 2 18446744071699945619 0 0 17 4 0 0 0
/proc/30746/statm: 12502 1736 480 364 0 1202 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 50008

[startup+0.301139 s]
/proc/loadavg: 1.31 2.98 3.55 2/386 30746
/proc/meminfo: memFree=21974964/32951124 swapFree=67111348/67111528
[pid=30746] ppid=30743 vsize=131292 CPUtime=0.18 cores=0,2,4,6
/proc/30746/stat : 30746 (python) R 30743 30746 30709 0 -1 4202496 5703 933 1 0 12 6 0 0 17 0 1 0 657899858 134443008 4098 33554432000 4194304 5683644 140735320068256 18446744073709551615 4335259 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/30746/statm: 32823 4098 1015 364 0 3084 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 131292

[startup+0.701067 s]
/proc/loadavg: 1.31 2.98 3.55 2/386 30746
/proc/meminfo: memFree=21974964/32951124 swapFree=67111348/67111528
[pid=30746] ppid=30743 vsize=246180 CPUtime=0.39 cores=0,2,4,6
/proc/30746/stat : 30746 (python) R 30743 30746 30709 0 -1 4202496 8123 933 1 0 26 13 0 0 16 0 1 0 657899858 252088320 6022 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687336133 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/30746/statm: 61545 6022 1621 364 0 4547 0
Current children cumulated CPU time (s) 0.39
Current children cumulated vsize (KiB) 246180

[startup+1.50096 s]
/proc/loadavg: 1.20 2.93 3.53 4/390 30758
/proc/meminfo: memFree=21911256/32951124 swapFree=67111348/67111528
[pid=30746] ppid=30743 vsize=290804 CPUtime=1.16 cores=0,2,4,6
/proc/30746/stat : 30746 (python) R 30743 30746 30709 0 -1 4202496 14010 933 1 0 100 16 0 0 17 0 1 0 657899858 297783296 10086 33554432000 4194304 5683644 140735320068256 18446744073709551615 212691084221 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/30746/statm: 72701 10086 1826 364 0 8625 0
Current children cumulated CPU time (s) 1.16
Current children cumulated vsize (KiB) 290804

[startup+3.10056 s]
/proc/loadavg: 1.20 2.93 3.53 4/390 30758
/proc/meminfo: memFree=21907164/32951124 swapFree=67111348/67111528
[pid=30746] ppid=30743 vsize=331056 CPUtime=2.27 cores=0,2,4,6
/proc/30746/stat : 30746 (python) S 30743 30746 30709 0 -1 4202496 16761 933 1 0 211 16 0 0 20 0 5 0 657899858 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/30746/statm: 82764 9955 1875 364 0 18673 0
Current children cumulated CPU time (s) 2.27
Current children cumulated vsize (KiB) 331056
heavy processes:

[startup+6.30098 s]
/proc/loadavg: 1.20 2.93 3.53 10/414 30782
/proc/meminfo: memFree=21770584/32951124 swapFree=67111348/67111528
[pid=30746] ppid=30743 vsize=331056 CPUtime=2.27 cores=0,2,4,6
/proc/30746/stat : 30746 (python) S 30743 30746 30709 0 -1 4202496 16761 933 1 0 211 16 0 0 20 0 5 0 657899858 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/30746/statm: 82764 9955 1875 364 0 18673 0
[pid=30771] ppid=30746 vsize=290044 CPUtime=0 cores=0,2,4,6
/proc/30771/stat : 30771 (python) S 30746 30746 30709 0 -1 4202560 719 0 0 0 0 0 0 0 18 0 1 0 657900117 297005056 8417 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695375 0 0 16781312 514 11066528462767358211 0 0 17 4 0 0 0
/proc/30771/statm: 72511 8417 353 364 0 8429 0
[pid=30773] ppid=30771 vsize=1020308 CPUtime=3.64 cores=0,2,4,6
/proc/30773/stat : 30773 (gnovelty+2) R 30771 30773 30773 0 -1 4202496 1154 0 0 0 364 0 0 0 25 0 1 0 657900117 1044795392 891 33554432000 134512640 135663704 4286982592 18446744073709551615 134544373 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30773/statm: 255077 891 113 282 0 254793 0
[pid=30774] ppid=30746 vsize=300300 CPUtime=0 cores=0,2,4,6
/proc/30774/stat : 30774 (python) S 30746 30746 30709 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 657900118 307507200 8415 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 7280433290043949315 0 0 17 0 0 0 0
/proc/30774/statm: 75075 8415 347 364 0 10990 0
[pid=30778] ppid=30774 vsize=1020308 CPUtime=3.66 cores=0,2,4,6
/proc/30778/stat : 30778 (gnovelty+2) R 30774 30778 30778 0 -1 4202496 1156 0 0 0 366 0 0 0 25 0 1 0 657900120 1044795392 892 33554432000 134512640 135663704 4287185744 18446744073709551615 134524561 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/30778/statm: 255077 892 113 282 0 254793 0
[pid=30776] ppid=30746 vsize=310556 CPUtime=0 cores=0,2,4,6
/proc/30776/stat : 30776 (python) S 30746 30746 30709 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 657900119 318009344 8419 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 7280429441753252097 0 0 17 0 0 0 0
/proc/30776/statm: 77639 8419 347 364 0 13551 0
[pid=30780] ppid=30776 vsize=423400 CPUtime=3.67 cores=0,2,4,6
/proc/30780/stat : 30780 (adaptg2wsat2009) R 30776 30780 30780 0 -1 4202496 2623 0 0 0 367 0 0 0 25 0 1 0 657900120 433561600 2357 33554432000 134512640 135107411 4292414592 18446744073709551615 134531781 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30780/statm: 105850 2357 59 146 0 104726 0
[pid=30779] ppid=30746 vsize=320812 CPUtime=0 cores=0,2,4,6
/proc/30779/stat : 30779 (python) S 30746 30746 30709 0 -1 4202560 743 0 0 0 0 0 0 0 18 0 1 0 657900120 328511488 8423 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 7280425593462554881 0 0 17 4 0 0 0
/proc/30779/statm: 80203 8423 347 364 0 16112 0
[pid=30782] ppid=30779 vsize=423400 CPUtime=3.66 cores=0,2,4,6
/proc/30782/stat : 30782 (adaptg2wsat2009) R 30779 30782 30782 0 -1 4202496 2620 0 0 0 366 0 0 0 25 0 1 0 657900121 433561600 2357 33554432000 134512640 135107411 4287552848 18446744073709551615 134531779 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/30782/statm: 105850 2357 59 146 0 104726 0
[pid=30746/tid=30772] ppid=30743 vsize=331056 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30772/stat : 30772 (python) S 30743 30746 30709 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 5 0 657900117 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30775] ppid=30743 vsize=331056 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30775/stat : 30775 (python) S 30743 30746 30709 0 -1 4202560 40 933 0 0 0 0 0 0 22 0 5 0 657900118 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 0 0 0 0
[pid=30746/tid=30777] ppid=30743 vsize=331056 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30777/stat : 30777 (python) S 30743 30746 30709 0 -1 4202560 16 933 0 0 0 0 0 0 24 0 5 0 657900120 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 0 0 0 0
[pid=30746/tid=30781] ppid=30743 vsize=331056 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30781/stat : 30781 (python) S 30743 30746 30709 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 5 0 657900121 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 16.9
Current children cumulated vsize (KiB) 4440184

[startup+12.7008 s]
/proc/loadavg: 2.62 3.18 3.60 10/414 30782
/proc/meminfo: memFree=21767136/32951124 swapFree=67111348/67111528
[pid=30746] ppid=30743 vsize=331056 CPUtime=2.27 cores=0,2,4,6
/proc/30746/stat : 30746 (python) S 30743 30746 30709 0 -1 4202496 16761 933 1 0 211 16 0 0 20 0 5 0 657899858 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/30746/statm: 82764 9955 1875 364 0 18673 0
[pid=30771] ppid=30746 vsize=290044 CPUtime=0.02 cores=0,2,4,6
/proc/30771/stat : 30771 (python) S 30746 30746 30709 0 -1 4202560 884 0 0 0 1 1 0 0 18 0 1 0 657900117 297005056 8440 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695375 0 0 16781312 514 5133100820597801219 0 0 17 4 0 0 0
/proc/30771/statm: 72511 8440 376 364 0 8429 0
[pid=30773] ppid=30771 vsize=1020308 CPUtime=9.95 cores=0,2,4,6
/proc/30773/stat : 30773 (gnovelty+2) R 30771 30773 30773 0 -1 4202496 1154 0 0 0 995 0 0 0 25 0 1 0 657900117 1044795392 891 33554432000 134512640 135663704 4286982592 18446744073709551615 134544196 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30773/statm: 255077 891 113 282 0 254793 0
[pid=30774] ppid=30746 vsize=300300 CPUtime=0.01 cores=0,2,4,6
/proc/30774/stat : 30774 (python) S 30746 30746 30709 0 -1 4202560 892 0 0 0 0 1 0 0 18 0 1 0 657900118 307507200 8438 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 7280433290043949315 0 0 17 0 0 0 0
/proc/30774/statm: 75075 8438 370 364 0 10990 0
[pid=30778] ppid=30774 vsize=1020308 CPUtime=10.06 cores=0,2,4,6
/proc/30778/stat : 30778 (gnovelty+2) R 30774 30778 30778 0 -1 4202496 1156 0 0 0 1006 0 0 0 25 0 1 0 657900120 1044795392 892 33554432000 134512640 135663704 4287185744 18446744073709551615 134524868 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/30778/statm: 255077 892 113 282 0 254793 0
[pid=30776] ppid=30746 vsize=310556 CPUtime=0.02 cores=0,2,4,6
/proc/30776/stat : 30776 (python) S 30746 30746 30709 0 -1 4202560 906 0 0 0 1 1 0 0 18 0 1 0 657900119 318009344 8442 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 5133100820597801217 0 0 17 0 0 0 0
/proc/30776/statm: 77639 8442 370 364 0 13551 0
[pid=30780] ppid=30776 vsize=423400 CPUtime=10 cores=0,2,4,6
/proc/30780/stat : 30780 (adaptg2wsat2009) R 30776 30780 30780 0 -1 4202496 2623 0 0 0 1000 0 0 0 25 0 1 0 657900120 433561600 2357 33554432000 134512640 135107411 4292414592 18446744073709551615 134531709 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30780/statm: 105850 2357 59 146 0 104726 0
[pid=30779] ppid=30746 vsize=320812 CPUtime=0.03 cores=0,2,4,6
/proc/30779/stat : 30779 (python) S 30746 30746 30709 0 -1 4202560 911 0 0 0 2 1 0 0 18 0 1 0 657900120 328511488 8446 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 5133100820597801217 0 0 17 4 0 0 0
/proc/30779/statm: 80203 8446 370 364 0 16112 0
[pid=30782] ppid=30779 vsize=423400 CPUtime=10.01 cores=0,2,4,6
/proc/30782/stat : 30782 (adaptg2wsat2009) R 30779 30782 30782 0 -1 4202496 2620 0 0 0 1001 0 0 0 25 0 1 0 657900121 433561600 2357 33554432000 134512640 135107411 4287552848 18446744073709551615 134532140 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/30782/statm: 105850 2357 59 146 0 104726 0
[pid=30746/tid=30772] ppid=30743 vsize=331056 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30772/stat : 30772 (python) S 30743 30746 30709 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 5 0 657900117 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30775] ppid=30743 vsize=331056 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30775/stat : 30775 (python) S 30743 30746 30709 0 -1 4202560 40 933 0 0 0 0 0 0 22 0 5 0 657900118 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 0 0 0 0
[pid=30746/tid=30777] ppid=30743 vsize=331056 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30777/stat : 30777 (python) S 30743 30746 30709 0 -1 4202560 16 933 0 0 0 0 0 0 24 0 5 0 657900120 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 0 0 0 0
[pid=30746/tid=30781] ppid=30743 vsize=331056 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30781/stat : 30781 (python) S 30743 30746 30709 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 5 0 657900121 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 42.37
Current children cumulated vsize (KiB) 4440184

[startup+25.5015 s]
/proc/loadavg: 3.45 3.34 3.65 9/414 30783
/proc/meminfo: memFree=21768988/32951124 swapFree=67111348/67111528
[pid=30746] ppid=30743 vsize=331056 CPUtime=2.27 cores=0,2,4,6
/proc/30746/stat : 30746 (python) S 30743 30746 30709 0 -1 4202496 16761 933 1 0 211 16 0 0 20 0 5 0 657899858 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/30746/statm: 82764 9955 1875 364 0 18673 0
[pid=30771] ppid=30746 vsize=290044 CPUtime=0.07 cores=0,2,4,6
/proc/30771/stat : 30771 (python) S 30746 30746 30709 0 -1 4202560 890 0 0 0 4 3 0 0 18 0 1 0 657900117 297005056 8440 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695375 0 0 16781312 514 5133100820597801219 0 0 17 4 0 0 0
/proc/30771/statm: 72511 8440 376 364 0 8429 0
[pid=30773] ppid=30771 vsize=1020308 CPUtime=22.56 cores=0,2,4,6
/proc/30773/stat : 30773 (gnovelty+2) R 30771 30773 30773 0 -1 4202496 1154 0 0 0 2256 0 0 0 25 0 1 0 657900117 1044795392 891 33554432000 134512640 135663704 4286982592 18446744073709551615 134564710 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30773/statm: 255077 891 113 282 0 254793 0
[pid=30774] ppid=30746 vsize=300300 CPUtime=0.06 cores=0,2,4,6
/proc/30774/stat : 30774 (python) S 30746 30746 30709 0 -1 4202560 905 0 0 0 3 3 0 0 18 0 1 0 657900118 307507200 8438 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 5133100820597801219 0 0 17 0 0 0 0
/proc/30774/statm: 75075 8438 370 364 0 10990 0
[pid=30778] ppid=30774 vsize=1020308 CPUtime=22.86 cores=0,2,4,6
/proc/30778/stat : 30778 (gnovelty+2) R 30774 30778 30778 0 -1 4202496 1156 0 0 0 2286 0 0 0 25 0 1 0 657900120 1044795392 892 33554432000 134512640 135663704 4287185744 18446744073709551615 134524688 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/30778/statm: 255077 892 113 282 0 254793 0
[pid=30776] ppid=30746 vsize=310556 CPUtime=0.06 cores=0,2,4,6
/proc/30776/stat : 30776 (python) S 30746 30746 30709 0 -1 4202560 911 0 0 0 3 3 0 0 18 0 1 0 657900119 318009344 8442 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 5133100820597801217 0 0 17 0 0 0 0
/proc/30776/statm: 77639 8442 370 364 0 13551 0
[pid=30780] ppid=30776 vsize=423400 CPUtime=22.73 cores=0,2,4,6
/proc/30780/stat : 30780 (adaptg2wsat2009) R 30776 30780 30780 0 -1 4202496 2623 0 0 0 2273 0 0 0 25 0 1 0 657900120 433561600 2357 33554432000 134512640 135107411 4292414592 18446744073709551615 134531561 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30780/statm: 105850 2357 59 146 0 104726 0
[pid=30779] ppid=30746 vsize=320812 CPUtime=0.06 cores=0,2,4,6
/proc/30779/stat : 30779 (python) S 30746 30746 30709 0 -1 4202560 915 0 0 0 3 3 0 0 18 0 1 0 657900120 328511488 8446 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 5133100820597801217 0 0 17 4 0 0 0
/proc/30779/statm: 80203 8446 370 364 0 16112 0
[pid=30782] ppid=30779 vsize=423400 CPUtime=22.72 cores=0,2,4,6
/proc/30782/stat : 30782 (adaptg2wsat2009) R 30779 30782 30782 0 -1 4202496 2620 0 0 0 2272 0 0 0 25 0 1 0 657900121 433561600 2357 33554432000 134512640 135107411 4287552848 18446744073709551615 134531600 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/30782/statm: 105850 2357 59 146 0 104726 0
[pid=30746/tid=30772] ppid=30743 vsize=331056 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30772/stat : 30772 (python) S 30743 30746 30709 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 5 0 657900117 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30775] ppid=30743 vsize=331056 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30775/stat : 30775 (python) S 30743 30746 30709 0 -1 4202560 40 933 0 0 0 0 0 0 22 0 5 0 657900118 339001344 9955 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 0 0 0 0
[pid=30746/tid=30777] ppid=30743 vsize=331056 CPUtime=0 cores=0,2,4,6

################
# More data... #
################

/proc/30780/stat : 30780 (adaptg2wsat2009) T 30776 30780 30780 0 -1 4202496 2623 0 0 0 9970 0 0 0 25 0 1 0 657900120 433561600 2357 33554432000 134512640 135107411 4292414592 18446744073709551615 134531600 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30780/statm: 105850 2357 59 146 0 104726 0
[pid=30776/tid=30793] ppid=30746 vsize=310556 CPUtime=0 cores=0,2,4,6
/proc/30776/task/30793/stat : 30793 (python) S 30746 30746 30709 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657910152 318009344 8472 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604446017865728 0 0 -1 4 0 0 0
[pid=30779] ppid=30746 vsize=320812 CPUtime=0.29 cores=0,2,4,6
/proc/30779/stat : 30779 (python) S 30746 30746 30709 0 -1 4202560 1094 0 0 0 14 15 0 0 18 0 2 0 657900120 328511488 8476 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/30779/statm: 80203 8476 399 364 0 16112 0
[pid=30782] ppid=30779 vsize=423400 CPUtime=99.69 cores=0,2,4,6
/proc/30782/stat : 30782 (adaptg2wsat2009) T 30779 30782 30782 0 -1 4202496 2620 0 0 0 9968 1 0 0 25 0 1 0 657900121 433561600 2357 33554432000 134512640 135107411 4287552848 18446744073709551615 134531239 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/30782/statm: 105850 2357 59 146 0 104726 0
[pid=30779/tid=30789] ppid=30746 vsize=320812 CPUtime=0 cores=0,2,4,6
/proc/30779/task/30789/stat : 30789 (python) S 30746 30746 30709 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657910150 328511488 8476 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604446017864832 0 0 -1 4 0 0 0
[pid=30790] ppid=30746 vsize=331068 CPUtime=0.3 cores=0,2,4,6
/proc/30790/stat : 30790 (python) S 30746 30746 30709 0 -1 4202560 1098 0 0 0 15 15 0 0 19 0 2 0 657910151 339013632 8480 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/30790/statm: 82767 8480 399 364 0 18673 0
[pid=30792] ppid=30790 vsize=423400 CPUtime=99.97 cores=0,2,4,6
/proc/30792/stat : 30792 (adaptg2wsat2009) T 30790 30792 30792 0 -1 4202496 2625 0 0 0 9997 0 0 0 25 0 1 0 657910151 433561600 2359 33554432000 134512640 135107411 4290839008 18446744073709551615 134531792 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30792/statm: 105850 2359 59 146 0 104726 0
[pid=30790/tid=30811] ppid=30746 vsize=331068 CPUtime=0 cores=0,2,4,6
/proc/30790/task/30811/stat : 30811 (python) S 30746 30746 30709 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 657920180 339013632 8480 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604449798535936 0 0 -1 4 0 0 0
[pid=30796] ppid=30746 vsize=341324 CPUtime=0.29 cores=0,2,4,6
/proc/30796/stat : 30796 (python) S 30746 30746 30709 0 -1 4202560 1094 0 0 0 14 15 0 0 19 0 2 0 657910154 349515776 8484 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/30796/statm: 85331 8484 399 364 0 21234 0
[pid=30800] ppid=30796 vsize=1020308 CPUtime=100.33 cores=0,2,4,6
/proc/30800/stat : 30800 (gnovelty+2) T 30796 30800 30800 0 -1 4202496 1151 0 0 0 10033 0 0 0 25 0 1 0 657910155 1044795392 891 33554432000 134512640 135663704 4291288320 18446744073709551615 134525691 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30800/statm: 255077 891 113 282 0 254793 0
[pid=30796/tid=30819] ppid=30746 vsize=341324 CPUtime=0 cores=0,2,4,6
/proc/30796/task/30819/stat : 30819 (python) S 30746 30746 30709 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657920193 349515776 8484 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604443318381248 0 0 -1 6 0 0 0
[pid=30798] ppid=30746 vsize=351580 CPUtime=0.3 cores=0,2,4,6
/proc/30798/stat : 30798 (python) S 30746 30746 30709 0 -1 4202560 1104 0 0 0 14 16 0 0 19 0 2 0 657910155 360017920 8488 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/30798/statm: 87895 8488 399 364 0 23795 0
[pid=30801] ppid=30798 vsize=253032 CPUtime=97.32 cores=0,2,4,6
/proc/30801/stat : 30801 (FH) T 30798 30801 30801 0 -1 4202496 2492 0 0 0 9732 0 0 0 25 0 1 0 657910156 259104768 2233 33554432000 134512640 134936988 4290439184 18446744073709551615 134554594 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/30801/statm: 63258 2233 41 104 0 62762 0
[pid=30798/tid=30815] ppid=30746 vsize=351580 CPUtime=0 cores=0,2,4,6
/proc/30798/task/30815/stat : 30815 (python) S 30746 30746 30709 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657920187 360017920 8488 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604450407809280 0 0 -1 6 0 0 0
[pid=30803] ppid=30746 vsize=361836 CPUtime=0.3 cores=0,2,4,6
/proc/30803/stat : 30803 (python) S 30746 30746 30709 0 -1 4202560 1135 0 0 0 15 15 0 0 19 0 2 0 657910163 370520064 8498 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/30803/statm: 90459 8498 405 364 0 26356 0
[pid=30805] ppid=30803 vsize=423400 CPUtime=100.01 cores=0,2,4,6
/proc/30805/stat : 30805 (adaptg2wsat2009) T 30803 30805 30805 0 -1 4202496 2627 0 0 0 10001 0 0 0 25 0 1 0 657910163 433561600 2357 33554432000 134512640 135107411 4290324752 18446744073709551615 134532019 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30805/statm: 105850 2357 59 146 0 104726 0
[pid=30803/tid=30820] ppid=30746 vsize=361836 CPUtime=0 cores=0,2,4,6
/proc/30803/task/30820/stat : 30820 (python) S 30746 30746 30709 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657920194 370520064 8498 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604446746958784 0 0 -1 2 0 0 0
[pid=30812] ppid=30746 vsize=372092 CPUtime=0.31 cores=0,2,4,6
/proc/30812/stat : 30812 (python) S 30746 30746 30709 0 -1 4202560 1151 0 0 0 19 12 0 0 19 0 2 0 657920181 381022208 8502 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/30812/statm: 93023 8502 405 364 0 28917 0
[pid=30814] ppid=30812 vsize=423400 CPUtime=99.96 cores=0,2,4,6
/proc/30814/stat : 30814 (adaptg2wsat2009) T 30812 30814 30814 0 -1 4202496 2629 0 0 0 9996 0 0 0 25 0 1 0 657920182 433561600 2357 33554432000 134512640 135107411 4290525472 18446744073709551615 134532107 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/30814/statm: 105850 2357 59 146 0 104726 0
[pid=30812/tid=30840] ppid=30746 vsize=372092 CPUtime=0 cores=0,2,4,6
/proc/30812/task/30840/stat : 30840 (python) S 30746 30746 30709 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657930212 381022208 8502 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604450305584128 0 0 -1 2 0 0 0
[pid=30816] ppid=30746 vsize=382348 CPUtime=0.31 cores=0,2,4,6
/proc/30816/stat : 30816 (python) S 30746 30746 30709 0 -1 4202560 1133 0 0 0 16 15 0 0 19 0 2 0 657920188 391524352 8500 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/30816/statm: 95587 8500 399 364 0 31478 0
[pid=30818] ppid=30816 vsize=253036 CPUtime=100 cores=0,2,4,6
/proc/30818/stat : 30818 (FH) T 30816 30818 30818 0 -1 4202496 2499 0 0 0 10000 0 0 0 25 0 1 0 657920188 259108864 2233 33554432000 134512640 134936988 4293253168 18446744073709551615 134552481 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/30818/statm: 63259 2233 41 104 0 62763 0
[pid=30816/tid=30844] ppid=30746 vsize=382348 CPUtime=0 cores=0,2,4,6
/proc/30816/task/30844/stat : 30844 (python) S 30746 30746 30709 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657930218 391524352 8500 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604453903501312 0 0 -1 6 0 0 0
[pid=30821] ppid=30746 vsize=392604 CPUtime=0.94 cores=0,2,4,6
/proc/30821/stat : 30821 (python) S 30746 30746 30709 0 -1 4202560 1187 0 0 0 52 42 0 0 18 0 2 0 657920195 402026496 8504 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/30821/statm: 98151 8504 399 364 0 34039 0
[pid=30823] ppid=30821 vsize=268428 CPUtime=293.58 cores=0,2,4,6
/proc/30823/stat : 30823 (hybridGM3) T 30821 30823 30823 0 -1 4202496 6183 0 0 0 29358 0 0 0 25 0 1 0 657920195 274870272 5919 33554432000 134512640 135204385 4291839920 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/30823/statm: 67107 5919 63 169 0 66937 0
[pid=30821/tid=30942] ppid=30746 vsize=392604 CPUtime=0 cores=0,2,4,6
/proc/30821/task/30942/stat : 30942 (python) S 30746 30746 30709 0 -1 4202560 50 0 0 0 0 0 0 0 18 0 2 0 657939774 402026496 8504 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604446899384704 0 0 -1 6 0 0 0
[pid=30824] ppid=30746 vsize=402860 CPUtime=0.3 cores=0,2,4,6
/proc/30824/stat : 30824 (python) S 30746 30746 30709 0 -1 4202560 1133 0 0 0 16 14 0 0 19 0 2 0 657920196 412528640 8508 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/30824/statm: 100715 8508 399 364 0 36600 0
[pid=30826] ppid=30824 vsize=1020308 CPUtime=97.28 cores=0,2,4,6
/proc/30826/stat : 30826 (gnovelty+2) T 30824 30826 30826 0 -1 4202496 1161 0 0 0 9728 0 0 0 25 0 1 0 657920196 1044795392 892 33554432000 134512640 135663704 4287545728 18446744073709551615 134524868 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/30826/statm: 255077 892 113 282 0 254793 0
[pid=30824/tid=30848] ppid=30746 vsize=402860 CPUtime=0 cores=0,2,4,6
/proc/30824/task/30848/stat : 30848 (python) S 30746 30746 30709 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657930233 412528640 8508 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604449456606208 0 0 -1 4 0 0 0
[pid=30841] ppid=30746 vsize=413116 CPUtime=0.62 cores=0,2,4,6
/proc/30841/stat : 30841 (python) S 30746 30746 30709 0 -1 4202560 1172 0 0 0 32 30 0 0 18 0 2 0 657930214 423030784 8512 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/30841/statm: 103279 8512 399 364 0 39161 0
[pid=30843] ppid=30841 vsize=268428 CPUtime=197.91 cores=0,2,4,6
/proc/30843/stat : 30843 (hybridGM3) R 30841 30843 30843 0 -1 4202496 6188 0 0 0 19788 3 0 0 25 0 1 0 657930214 274870272 5920 33554432000 134512640 135204385 4292138480 18446744073709551615 134538061 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/30843/statm: 67107 5920 63 169 0 66937 0
[pid=30841/tid=30952] ppid=30746 vsize=413116 CPUtime=0 cores=0,2,4,6
/proc/30841/task/30952/stat : 30952 (python) S 30746 30746 30709 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657949883 423030784 8512 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604444479336832 0 0 -1 2 0 0 0
[pid=30845] ppid=30746 vsize=423372 CPUtime=0.32 cores=0,2,4,6
/proc/30845/stat : 30845 (python) S 30746 30746 30709 0 -1 4202560 1164 0 0 0 18 14 0 0 19 0 2 0 657930220 433532928 8516 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/30845/statm: 105843 8516 399 364 0 41722 0
[pid=30847] ppid=30845 vsize=423400 CPUtime=97.43 cores=0,2,4,6
/proc/30847/stat : 30847 (adaptg2wsat2009) T 30845 30847 30847 0 -1 4202496 2629 0 0 0 9742 1 0 0 25 0 1 0 657930220 433561600 2357 33554432000 134512640 135107411 4292463344 18446744073709551615 134532128 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/30847/statm: 105850 2357 59 146 0 104726 0
[pid=30845/tid=30943] ppid=30746 vsize=423372 CPUtime=0 cores=0,2,4,6
/proc/30845/task/30943/stat : 30943 (python) S 30746 30746 30709 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657940251 433532928 8516 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604444479337728 0 0 -1 6 0 0 0
[pid=30849] ppid=30746 vsize=433628 CPUtime=0.62 cores=0,2,4,6
/proc/30849/stat : 30849 (python) S 30746 30746 30709 0 -1 4202560 1235 0 0 0 33 29 0 0 19 0 2 0 657930234 444035072 8528 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/30849/statm: 108407 8528 407 364 0 44283 0
[pid=30851] ppid=30849 vsize=268420 CPUtime=195.77 cores=0,2,4,6
/proc/30851/stat : 30851 (hybridGM3) T 30849 30851 30851 0 -1 4202496 6191 0 0 0 19576 1 0 0 25 0 1 0 657930234 274862080 5918 33554432000 134512640 135204385 4289441456 18446744073709551615 134538541 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/30851/statm: 67105 5918 63 169 0 66935 0
[pid=30849/tid=30953] ppid=30746 vsize=433628 CPUtime=0 cores=0,2,4,6
/proc/30849/task/30953/stat : 30953 (python) S 30746 30746 30709 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657949903 444035072 8528 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 514 18446604451151405120 0 0 -1 0 0 0 0
[pid=30944] ppid=30746 vsize=443884 CPUtime=0.3 cores=0,2,4,6
/proc/30944/stat : 30944 (python) S 30746 30746 30709 0 -1 4202560 1040 0 0 0 16 14 0 0 18 0 1 0 657940252 454537216 8501 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 5133100820597801218 0 0 17 4 0 0 0
/proc/30944/statm: 110971 8501 377 364 0 46844 0
[pid=30946] ppid=30944 vsize=268424 CPUtime=96.65 cores=0,2,4,6
/proc/30946/stat : 30946 (hybridGM3) R 30944 30946 30946 0 -1 4202496 6192 0 0 0 9665 0 0 0 25 0 1 0 657940254 274866176 5920 33554432000 134512640 135204385 4287893936 18446744073709551615 134538355 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/30946/statm: 67106 5920 63 169 0 66936 0
[pid=30954] ppid=30746 vsize=454140 CPUtime=0 cores=0,2,4,6
/proc/30954/stat : 30954 (python) S 30746 30746 30709 0 -1 4202560 854 0 0 0 0 0 0 0 18 0 1 0 657949904 465039360 8482 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 7249291822210482175 0 0 17 4 0 0 0
/proc/30954/statm: 113535 8482 354 364 0 49405 0
[pid=30956] ppid=30954 vsize=461508 CPUtime=1.06 cores=0,2,4,6
/proc/30956/stat : 30956 (TNM) R 30954 30956 30956 0 -1 4202496 2468 0 0 0 106 0 0 0 24 0 1 0 657949904 472584192 2195 33554432000 134512640 135008748 4293977840 18446744073709551615 134527788 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30956/statm: 115377 2195 46 122 0 114277 0
[pid=30957] ppid=30746 vsize=464396 CPUtime=0 cores=0,2,4,6
/proc/30957/stat : 30957 (python) S 30746 30746 30709 0 -1 4202560 840 0 0 0 0 0 0 0 18 0 1 0 657949910 475541504 8480 33554432000 4194304 5683644 140735320068256 18446744073709551615 212687695430 0 0 16781312 514 426981051290615807 0 0 17 4 0 0 0
/proc/30957/statm: 116099 8480 348 364 0 51966 0
[pid=30959] ppid=30957 vsize=253032 CPUtime=0.95 cores=0,2,4,6
/proc/30959/stat : 30959 (FH) R 30957 30959 30959 0 -1 4202496 2508 0 0 0 95 0 0 0 25 0 1 0 657949910 259104768 2233 33554432000 134512640 134936988 4291379424 18446744073709551615 134554097 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30959/statm: 63258 2233 41 104 0 62762 0
[pid=30746/tid=30772] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30772/stat : 30772 (python) S 30743 30746 30709 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 19 0 657900117 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30775] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30775/stat : 30775 (python) S 30743 30746 30709 0 -1 4202560 40 933 0 0 0 0 0 0 22 0 19 0 657900118 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 0 0 0 0
[pid=30746/tid=30777] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30777/stat : 30777 (python) S 30743 30746 30709 0 -1 4202560 16 933 0 0 0 0 0 0 24 0 19 0 657900120 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 0 0 0 0
[pid=30746/tid=30781] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30781/stat : 30781 (python) S 30743 30746 30709 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 19 0 657900121 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30791] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30791/stat : 30791 (python) S 30743 30746 30709 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 19 0 657910151 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30797] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30797/stat : 30797 (python) S 30743 30746 30709 0 -1 4202560 36 933 0 0 0 0 0 0 21 0 19 0 657910154 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 0 0 0 0
[pid=30746/tid=30799] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30799/stat : 30799 (python) S 30743 30746 30709 0 -1 4202560 19 933 0 0 0 0 0 0 23 0 19 0 657910155 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 0 0 0 0
[pid=30746/tid=30804] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30804/stat : 30804 (python) S 30743 30746 30709 0 -1 4202560 26 933 0 0 0 0 0 0 24 0 19 0 657910163 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 2 0 0 0
[pid=30746/tid=30813] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30813/stat : 30813 (python) S 30743 30746 30709 0 -1 4202560 22 933 0 0 0 0 0 0 20 0 19 0 657920182 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 2 0 0 0
[pid=30746/tid=30817] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30817/stat : 30817 (python) S 30743 30746 30709 0 -1 4202560 27 933 0 0 0 0 0 0 21 0 19 0 657920188 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 6 0 0 0
[pid=30746/tid=30822] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30822/stat : 30822 (python) S 30743 30746 30709 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 19 0 657920195 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 6 0 0 0
[pid=30746/tid=30825] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30825/stat : 30825 (python) S 30743 30746 30709 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 19 0 657920196 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30842] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30842/stat : 30842 (python) S 30743 30746 30709 0 -1 4202560 31 933 0 0 0 0 0 0 18 0 19 0 657930214 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30846] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30846/stat : 30846 (python) S 30743 30746 30709 0 -1 4202560 30 933 0 0 0 0 0 0 21 0 19 0 657930220 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30850] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30850/stat : 30850 (python) S 30743 30746 30709 0 -1 4202560 25 933 0 0 0 0 0 0 21 0 19 0 657930234 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30945] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30945/stat : 30945 (python) S 30743 30746 30709 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 19 0 657940252 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30955] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30955/stat : 30955 (python) S 30743 30746 30709 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 19 0 657949904 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
[pid=30746/tid=30958] ppid=30743 vsize=474640 CPUtime=0 cores=0,2,4,6
/proc/30746/task/30958/stat : 30958 (python) S 30743 30746 30709 0 -1 4202560 29 933 0 0 0 0 0 0 21 0 19 0 657949910 486031360 10048 33554432000 4194304 5683644 140735320068256 18446744073709551615 212699498929 0 0 16781312 2 18446604446318757184 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1985.52
Current children cumulated vsize (KiB) 16190784

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 30746 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=458626
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=375942
# CPU time returned by wait4() is 2.83457
# while last known CPU time is 1985.52
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 501.635
CPU time (s): 1985.52
CPU user time (s): 1982.37
CPU system time (s): 3.15
CPU usage (%): 395.81
Max. virtual memory (cumulated for all children) (KiB): 16190784

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.45863
system time used= 0.375942
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 32485
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 14843
involuntary context switches= 55

runsolver used 3.46847 second user time and 8.5137 second system time

The end

Launcher Data 

Begin job on node139 at 2011-05-15 14:37:01
IDJOB=3378134
IDBENCH=82543
IDSOLVER=1859
FILE ID=node139/3378134-1305463021
RUNJOBID= node139-1305463021-30728
PBS_JOBID= 13324978
Free space on /tmp= 73400 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k5-r20-v2000-c40000-S319509287-085-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378134-1305463021/watcher-3378134-1305463021 -o /tmp/evaluation-result-3378134-1305463021/solver-3378134-1305463021 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378134-1305463021.cnf -seed 1937025153 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= bb548a1e884feefbb349d3449e849f5e
RANDOM SEED=1937025153

node139.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5321.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21975640 kB
Buffers:        318412 kB
Cached:        3066860 kB
SwapCached:          0 kB
Active:        8602428 kB
Inactive:      2060776 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21975640 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:           65376 kB
Writeback:           0 kB
AnonPages:     7277884 kB
Mapped:          30140 kB
Slab:           205100 kB
PageTables:      40308 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 81746072 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73332 MiB
End job on node139 at 2011-05-15 14:45:25