Trace number 3317155

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 2006.63 506.022

General information on the benchmark

Namerandom/large/
unif-k7-r85-v400-c34000-S943787219-031-UNKNOWN.cnf
MD5SUM6105ec111e1e16da586afadcc6f09041
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables400
Number of clauses34000
Sum of the clauses size238000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 534000

Solver Data 

0.29/0.49	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.29/0.49	c INFO: solving HOME/instance-3317155-1304267811.cnf
0.29/0.49	c INFO: computing task features
0.58/0.72	c INFO: computed variable-clause graph statistics
0.58/0.73	c INFO: computed clause balance statistics
1.77/1.95	c INFO: computed variable balance statistics
2.27/2.43	c INFO: computed small-clause counts
2.27/2.44	c INFO: computed Horn-clause counts
2.27/2.48	c INFO: computed variable graph statistics
2.27/2.48	c INFO: collected features for HOME/instance-3317155-1304267811.cnf in 1.98 s
2.38/2.51	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.38)
2.38/2.53	c INFO: running gnovelty+2@0 for 97 with 4797 remaining (b = 0.13)
2.38/2.56	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.08)
2.38/2.59	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.07)
402.24/102.73	c INFO: running adaptg2wsat2009++@0 for 97 with 4697 remaining (b = 0.05)
402.24/102.77	c INFO: running gnovelty+2@0 for 97 with 4697 remaining (b = 0.04)
402.50/102.81	c INFO: running FH@0 for 97 with 4697 remaining (b = 0.03)
402.50/102.85	c INFO: running adaptg2wsat2009++@0 for 97 with 4697 remaining (b = 0.03)
801.26/202.92	c INFO: running adaptg2wsat2009++@0 for 97 with 4597 remaining (b = 0.02)
801.60/203.02	c INFO: running FH@0 for 97 with 4597 remaining (b = 0.02)
801.60/203.07	c INFO: running hybridGM3@0 for 194 with 4597 remaining (b = 0.04)
801.77/203.12	c INFO: running gnovelty+2@0 for 97 with 4597 remaining (b = 0.02)
1196.27/302.13	c INFO: running hybridGM3@0 for 194 with 4498 remaining (b = 0.03)
1200.62/303.23	c INFO: running adaptg2wsat2009++@0 for 97 with 4497 remaining (b = 0.01)
1200.95/303.38	c INFO: running hybridGM3@0 for 194 with 4497 remaining (b = 0.03)
1583.90/399.53	c INFO: running hybridGM3@98 for 97 with 4400 remaining (b = 0.01)
1599.42/403.46	c INFO: running hybridGM3@0 for 194 with 4397 remaining (b = 0.02)
1962.11/494.74	c INFO: running TNM@0 for 1552 with 4305 remaining (b = 0.18)
1977.59/498.61	c INFO: running hybridGM3@100 for 97 with 4301 remaining (b = 0.01)
2006.19/505.87	c INFO: running FH@0 for 97 with 4294 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317155-1304267811/watcher-3317155-1304267811 -o /tmp/evaluation-result-3317155-1304267811/solver-3317155-1304267811 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317155-1304267811.cnf -seed 1317240943 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.66 6.55 3.37 4/211 10156
/proc/meminfo: memFree=30478340/32951124 swapFree=67106388/67111528
[pid=10156] ppid=10154 vsize=15756 CPUtime=0 cores=1,3,5,7
/proc/10156/stat : 10156 (python) S 10154 10156 9843 0 -1 4202496 839 0 0 0 0 0 0 0 22 0 1 0 519197418 16134144 723 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039653376 0 2147483391 16781312 2 18446744071699199166 0 0 17 3 0 0 0
/proc/10156/statm: 3939 723 334 364 0 426 0

[startup+0.0837849 s]
/proc/loadavg: 7.66 6.55 3.37 4/211 10156
/proc/meminfo: memFree=30478340/32951124 swapFree=67106388/67111528
[pid=10156] ppid=10154 vsize=85364 CPUtime=0.05 cores=1,3,5,7
/proc/10156/stat : 10156 (python) D 10154 10156 9843 0 -1 4202496 2583 0 0 0 4 1 0 0 22 0 1 0 519197418 87412736 2308 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039661111 0 0 16781312 2 18446744071562154983 0 0 17 3 0 0 0
/proc/10156/statm: 21341 2308 707 364 0 1558 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 85364

[startup+0.100793 s]
/proc/loadavg: 7.66 6.55 3.37 4/211 10156
/proc/meminfo: memFree=30478340/32951124 swapFree=67106388/67111528
[pid=10156] ppid=10154 vsize=85364 CPUtime=0.05 cores=1,3,5,7
/proc/10156/stat : 10156 (python) D 10154 10156 9843 0 -1 4202496 2583 0 0 0 4 1 0 0 22 0 1 0 519197418 87412736 2308 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039661111 0 0 16781312 2 18446744071562154983 0 0 17 3 0 0 0
/proc/10156/statm: 21341 2308 707 364 0 1558 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 85364

[startup+0.300709 s]
/proc/loadavg: 7.66 6.55 3.37 4/211 10156
/proc/meminfo: memFree=30478340/32951124 swapFree=67106388/67111528
[pid=10156] ppid=10154 vsize=135900 CPUtime=0.22 cores=1,3,5,7
/proc/10156/stat : 10156 (python) S 10154 10156 9843 0 -1 4202496 6040 935 1 0 16 6 0 0 18 0 1 0 519197418 139161600 4187 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039653376 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 2
/proc/10156/statm: 33975 4187 1032 364 0 3161 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 135900

[startup+0.700671 s]
/proc/loadavg: 7.66 6.55 3.37 4/211 10156
/proc/meminfo: memFree=30478340/32951124 swapFree=67106388/67111528
[pid=10156] ppid=10154 vsize=288804 CPUtime=0.58 cores=1,3,5,7
/proc/10156/stat : 10156 (python) R 10154 10156 9843 0 -1 4202496 11696 935 1 0 49 9 0 0 18 0 1 0 519197418 295735296 9376 33554432000 4194304 5683644 140734217194144 18446744073709551615 47854224812357 0 0 16781312 2 0 0 0 17 1 0 0 2
/proc/10156/statm: 72201 9376 1761 364 0 8125 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 288804

[startup+1.50052 s]
/proc/loadavg: 7.66 6.55 3.37 6/218 10167
/proc/meminfo: memFree=30330604/32951124 swapFree=67106388/67111528
[pid=10156] ppid=10154 vsize=289500 CPUtime=1.37 cores=1,3,5,7
/proc/10156/stat : 10156 (python) R 10154 10156 9843 0 -1 4202496 13884 935 1 0 128 9 0 0 20 0 1 0 519197418 296448000 9759 33554432000 4194304 5683644 140734217194144 18446744073709551615 4562986 0 0 16781312 2 0 0 0 17 1 0 0 2
/proc/10156/statm: 72375 9759 1826 364 0 8299 0
Current children cumulated CPU time (s) 1.37
Current children cumulated vsize (KiB) 289500

[startup+3.10021 s]
/proc/loadavg: 7.66 6.55 3.37 6/218 10167
/proc/meminfo: memFree=30329860/32951124 swapFree=67106388/67111528
[pid=10156] ppid=10154 vsize=323404 CPUtime=2.48 cores=1,3,5,7
/proc/10156/stat : 10156 (python) S 10154 10156 9843 0 -1 4202496 16745 935 1 0 237 11 0 0 20 0 5 0 519197418 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 2
/proc/10156/statm: 80851 8049 1883 364 0 16760 0
Current children cumulated CPU time (s) 2.48
Current children cumulated vsize (KiB) 323404
heavy processes:

[startup+6.30064 s]
/proc/loadavg: 7.45 6.53 3.38 9/230 10179
/proc/meminfo: memFree=30301880/32951124 swapFree=67106388/67111528
[pid=10156] ppid=10154 vsize=323404 CPUtime=2.48 cores=1,3,5,7
/proc/10156/stat : 10156 (python) S 10154 10156 9843 0 -1 4202496 16745 935 1 0 237 11 0 0 20 0 5 0 519197418 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 2
/proc/10156/statm: 80851 8049 1883 364 0 16760 0
[pid=10168] ppid=10156 vsize=282244 CPUtime=0 cores=1,3,5,7
/proc/10168/stat : 10168 (python) S 10156 10156 9843 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 519197669 289017856 6331 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675407 0 0 16781312 514 4588148777306521864 0 0 17 1 0 0 0
/proc/10168/statm: 70561 6331 355 364 0 6479 0
[pid=10170] ppid=10168 vsize=1020188 CPUtime=3.75 cores=1,3,5,7
/proc/10170/stat : 10170 (gnovelty+2) R 10168 10170 10170 0 -1 4202496 1145 0 0 0 375 0 0 0 25 0 1 0 519197669 1044672512 877 33554432000 134512640 135663704 4291107056 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/10170/statm: 255047 877 113 282 0 254763 0
[pid=10171] ppid=10156 vsize=293132 CPUtime=0 cores=1,3,5,7
/proc/10171/stat : 10171 (python) S 10156 10156 9843 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 519197672 300167168 6369 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 4523083252587528454 0 0 17 7 0 0 0
/proc/10171/statm: 73283 6369 349 364 0 9198 0
[pid=10173] ppid=10171 vsize=1020188 CPUtime=3.75 cores=1,3,5,7
/proc/10173/stat : 10173 (gnovelty+2) R 10171 10173 10173 0 -1 4202496 1141 0 0 0 375 0 0 0 25 0 1 0 519197672 1044672512 877 33554432000 134512640 135663704 4289721264 18446744073709551615 134545619 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10173/statm: 255047 877 113 282 0 254763 0
[pid=10174] ppid=10156 vsize=302904 CPUtime=0 cores=1,3,5,7
/proc/10174/stat : 10174 (python) S 10156 10156 9843 0 -1 4202560 735 0 0 0 0 0 0 0 18 0 1 0 519197675 310173696 6506 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 13997552134267044101 0 0 17 1 0 0 0
/proc/10174/statm: 75726 6506 347 364 0 11638 0
[pid=10176] ppid=10174 vsize=421140 CPUtime=3.72 cores=1,3,5,7
/proc/10176/stat : 10176 (adaptg2wsat2009) R 10174 10176 10176 0 -1 4202496 1802 0 0 0 372 0 0 0 25 0 1 0 519197675 431247360 1538 33554432000 134512640 135107411 4291662976 18446744073709551615 134523331 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/10176/statm: 105285 1538 49 146 0 104161 0
[pid=10177] ppid=10156 vsize=313160 CPUtime=0 cores=1,3,5,7
/proc/10177/stat : 10177 (python) S 10156 10156 9843 0 -1 4202560 746 0 0 0 0 0 0 0 18 0 1 0 519197678 320675840 6510 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 3270409655062331656 0 0 17 7 0 0 0
/proc/10177/statm: 78290 6510 347 364 0 14199 0
[pid=10179] ppid=10177 vsize=421140 CPUtime=3.69 cores=1,3,5,7
/proc/10179/stat : 10179 (adaptg2wsat2009) R 10177 10179 10179 0 -1 4202496 1805 0 0 0 369 0 0 0 25 0 1 0 519197678 431247360 1538 33554432000 134512640 135107411 4287306464 18446744073709551615 134523245 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10179/statm: 105285 1538 49 146 0 104161 0
[pid=10156/tid=10169] ppid=10154 vsize=323404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10169/stat : 10169 (python) S 10154 10156 9843 0 -1 4202560 26 935 0 0 0 0 0 0 24 0 5 0 519197669 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 1 0 0 0
[pid=10156/tid=10172] ppid=10154 vsize=323404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10172/stat : 10172 (python) S 10154 10156 9843 0 -1 4202560 24 935 0 0 0 0 0 0 25 0 5 0 519197672 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10175] ppid=10154 vsize=323404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10175/stat : 10175 (python) S 10154 10156 9843 0 -1 4202560 23 935 0 0 0 0 0 0 20 0 5 0 519197675 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 1 0 0 0
[pid=10156/tid=10178] ppid=10154 vsize=323404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10178/stat : 10178 (python) S 10154 10156 9843 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 5 0 519197678 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 17.39
Current children cumulated vsize (KiB) 4397500

[startup+12.7015 s]
/proc/loadavg: 7.49 6.55 3.41 9/230 10179
/proc/meminfo: memFree=30298148/32951124 swapFree=67106388/67111528
[pid=10156] ppid=10154 vsize=323404 CPUtime=2.48 cores=1,3,5,7
/proc/10156/stat : 10156 (python) S 10154 10156 9843 0 -1 4202496 16745 935 1 0 237 11 0 0 20 0 5 0 519197418 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 2
/proc/10156/statm: 80851 8049 1883 364 0 16760 0
[pid=10168] ppid=10156 vsize=282244 CPUtime=0.01 cores=1,3,5,7
/proc/10168/stat : 10168 (python) S 10156 10156 9843 0 -1 4202560 899 0 0 0 0 1 0 0 18 0 1 0 519197669 289017856 6354 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675407 0 0 16781312 514 16935645661238362376 0 0 17 1 0 0 0
/proc/10168/statm: 70561 6354 378 364 0 6479 0
[pid=10170] ppid=10168 vsize=1020188 CPUtime=10.07 cores=1,3,5,7
/proc/10170/stat : 10170 (gnovelty+2) R 10168 10170 10170 0 -1 4202496 1145 0 0 0 1007 0 0 0 25 0 1 0 519197669 1044672512 877 33554432000 134512640 135663704 4291107056 18446744073709551615 134525506 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/10170/statm: 255047 877 113 282 0 254763 0
[pid=10171] ppid=10156 vsize=293132 CPUtime=0.01 cores=1,3,5,7
/proc/10171/stat : 10171 (python) S 10156 10156 9843 0 -1 4202560 910 0 0 0 1 0 0 0 18 0 1 0 519197672 300167168 6392 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 16935645661238362374 0 0 17 7 0 0 0
/proc/10171/statm: 73283 6392 372 364 0 9198 0
[pid=10173] ppid=10171 vsize=1020188 CPUtime=10.15 cores=1,3,5,7
/proc/10173/stat : 10173 (gnovelty+2) R 10171 10173 10173 0 -1 4202496 1141 0 0 0 1015 0 0 0 25 0 1 0 519197672 1044672512 877 33554432000 134512640 135663704 4289721264 18446744073709551615 134525699 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10173/statm: 255047 877 113 282 0 254763 0
[pid=10174] ppid=10156 vsize=302904 CPUtime=0.01 cores=1,3,5,7
/proc/10174/stat : 10174 (python) S 10156 10156 9843 0 -1 4202560 911 0 0 0 0 1 0 0 18 0 1 0 519197675 310173696 6529 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 16935645661238362373 0 0 17 1 0 0 0
/proc/10174/statm: 75726 6529 370 364 0 11638 0
[pid=10176] ppid=10174 vsize=421804 CPUtime=10.08 cores=1,3,5,7
/proc/10176/stat : 10176 (adaptg2wsat2009) R 10174 10176 10176 0 -1 4202496 2134 0 0 0 1008 0 0 0 25 0 1 0 519197675 431927296 1870 33554432000 134512640 135107411 4291662976 18446744073709551615 134531239 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/10176/statm: 105451 1870 59 146 0 104327 0
[pid=10177] ppid=10156 vsize=313160 CPUtime=0.01 cores=1,3,5,7
/proc/10177/stat : 10177 (python) S 10156 10156 9843 0 -1 4202560 910 0 0 0 0 1 0 0 18 0 1 0 519197678 320675840 6533 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 16935645661238362376 0 0 17 7 0 0 0
/proc/10177/statm: 78290 6533 370 364 0 14199 0
[pid=10179] ppid=10177 vsize=421804 CPUtime=10.09 cores=1,3,5,7
/proc/10179/stat : 10179 (adaptg2wsat2009) R 10177 10179 10179 0 -1 4202496 2138 0 0 0 1009 0 0 0 25 0 1 0 519197678 431927296 1871 33554432000 134512640 135107411 4287306464 18446744073709551615 134532135 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10179/statm: 105451 1871 59 146 0 104327 0
[pid=10156/tid=10169] ppid=10154 vsize=323404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10169/stat : 10169 (python) S 10154 10156 9843 0 -1 4202560 26 935 0 0 0 0 0 0 24 0 5 0 519197669 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 1 0 0 0
[pid=10156/tid=10172] ppid=10154 vsize=323404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10172/stat : 10172 (python) S 10154 10156 9843 0 -1 4202560 24 935 0 0 0 0 0 0 25 0 5 0 519197672 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10175] ppid=10154 vsize=323404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10175/stat : 10175 (python) S 10154 10156 9843 0 -1 4202560 23 935 0 0 0 0 0 0 20 0 5 0 519197675 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 1 0 0 0
[pid=10156/tid=10178] ppid=10154 vsize=323404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10178/stat : 10178 (python) S 10154 10156 9843 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 5 0 519197678 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 42.91
Current children cumulated vsize (KiB) 4398828

[startup+25.5012 s]
/proc/loadavg: 7.61 6.62 3.48 9/230 10179
/proc/meminfo: memFree=30298392/32951124 swapFree=67106388/67111528
[pid=10156] ppid=10154 vsize=323404 CPUtime=2.48 cores=1,3,5,7
/proc/10156/stat : 10156 (python) S 10154 10156 9843 0 -1 4202496 16745 935 1 0 237 11 0 0 20 0 5 0 519197418 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 2
/proc/10156/statm: 80851 8049 1883 364 0 16760 0
[pid=10168] ppid=10156 vsize=282244 CPUtime=0.03 cores=1,3,5,7
/proc/10168/stat : 10168 (python) S 10156 10156 9843 0 -1 4202560 905 0 0 0 1 2 0 0 18 0 1 0 519197669 289017856 6354 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675407 0 0 16781312 514 16935645661238362376 0 0 17 1 0 0 0
/proc/10168/statm: 70561 6354 378 364 0 6479 0
[pid=10170] ppid=10168 vsize=1020188 CPUtime=22.75 cores=1,3,5,7
/proc/10170/stat : 10170 (gnovelty+2) R 10168 10170 10170 0 -1 4202496 1145 0 0 0 2275 0 0 0 25 0 1 0 519197669 1044672512 877 33554432000 134512640 135663704 4291107056 18446744073709551615 134525213 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/10170/statm: 255047 877 113 282 0 254763 0
[pid=10171] ppid=10156 vsize=293132 CPUtime=0.03 cores=1,3,5,7
/proc/10171/stat : 10171 (python) S 10156 10156 9843 0 -1 4202560 914 0 0 0 2 1 0 0 18 0 1 0 519197672 300167168 6392 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 16935645661238362374 0 0 17 7 0 0 0
/proc/10171/statm: 73283 6392 372 364 0 9198 0
[pid=10173] ppid=10171 vsize=1020188 CPUtime=22.95 cores=1,3,5,7
/proc/10173/stat : 10173 (gnovelty+2) R 10171 10173 10173 0 -1 4202496 1141 0 0 0 2295 0 0 0 25 0 1 0 519197672 1044672512 877 33554432000 134512640 135663704 4289721264 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10173/statm: 255047 877 113 282 0 254763 0
[pid=10174] ppid=10156 vsize=302904 CPUtime=0.03 cores=1,3,5,7
/proc/10174/stat : 10174 (python) S 10156 10156 9843 0 -1 4202560 916 0 0 0 1 2 0 0 18 0 1 0 519197675 310173696 6529 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 16935645661238362373 0 0 17 1 0 0 0
/proc/10174/statm: 75726 6529 370 364 0 11638 0
[pid=10176] ppid=10174 vsize=421804 CPUtime=22.83 cores=1,3,5,7
/proc/10176/stat : 10176 (adaptg2wsat2009) R 10174 10176 10176 0 -1 4202496 2134 0 0 0 2283 0 0 0 25 0 1 0 519197675 431927296 1870 33554432000 134512640 135107411 4291662976 18446744073709551615 134532128 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/10176/statm: 105451 1870 59 146 0 104327 0
[pid=10177] ppid=10156 vsize=313160 CPUtime=0.03 cores=1,3,5,7
/proc/10177/stat : 10177 (python) S 10156 10156 9843 0 -1 4202560 915 0 0 0 1 2 0 0 18 0 1 0 519197678 320675840 6533 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 16935645661238362376 0 0 17 1 0 0 0
/proc/10177/statm: 78290 6533 370 364 0 14199 0
[pid=10179] ppid=10177 vsize=421804 CPUtime=22.89 cores=1,3,5,7
/proc/10179/stat : 10179 (adaptg2wsat2009) R 10177 10179 10179 0 -1 4202496 2138 0 0 0 2289 0 0 0 25 0 1 0 519197678 431927296 1871 33554432000 134512640 135107411 4287306464 18446744073709551615 134531246 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10179/statm: 105451 1871 59 146 0 104327 0
[pid=10156/tid=10169] ppid=10154 vsize=323404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10169/stat : 10169 (python) S 10154 10156 9843 0 -1 4202560 26 935 0 0 0 0 0 0 24 0 5 0 519197669 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 1 0 0 0
[pid=10156/tid=10172] ppid=10154 vsize=323404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10172/stat : 10172 (python) S 10154 10156 9843 0 -1 4202560 24 935 0 0 0 0 0 0 25 0 5 0 519197672 331165696 8049 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10175] ppid=10154 vsize=323404 CPUtime=0 cores=1,3,5,7

################
# More data... #
################

/proc/10176/stat : 10176 (adaptg2wsat2009) T 10174 10176 10176 0 -1 4202496 2134 0 0 0 9972 0 0 0 25 0 1 0 519197675 431927296 1870 33554432000 134512640 135107411 4291662976 18446744073709551615 134531600 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10176/statm: 105451 1870 59 146 0 104327 0
[pid=10174/tid=10188] ppid=10156 vsize=302904 CPUtime=0 cores=1,3,5,7
/proc/10174/task/10188/stat : 10188 (python) S 10156 10156 9843 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 519207691 310173696 6559 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604469056654208 0 0 -1 1 0 0 0
[pid=10177] ppid=10156 vsize=313160 CPUtime=0.15 cores=1,3,5,7
/proc/10177/stat : 10177 (python) S 10156 10156 9843 0 -1 4202560 1103 0 0 0 7 8 0 0 19 0 2 0 519197678 320675840 6563 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/10177/statm: 78290 6563 399 364 0 14199 0
[pid=10179] ppid=10177 vsize=421804 CPUtime=100.14 cores=1,3,5,7
/proc/10179/stat : 10179 (adaptg2wsat2009) T 10177 10179 10179 0 -1 4202496 2138 0 0 0 10014 0 0 0 25 0 1 0 519197678 431927296 1871 33554432000 134512640 135107411 4287306464 18446744073709551615 134531942 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10179/statm: 105451 1871 59 146 0 104327 0
[pid=10177/tid=10192] ppid=10156 vsize=313160 CPUtime=0 cores=1,3,5,7
/proc/10177/task/10192/stat : 10192 (python) S 10156 10156 9843 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 519207693 320675840 6563 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604462264079424 0 0 -1 1 0 0 0
[pid=10189] ppid=10156 vsize=323612 CPUtime=0.16 cores=1,3,5,7
/proc/10189/stat : 10189 (python) S 10156 10156 9843 0 -1 4202560 1129 0 0 0 8 8 0 0 19 0 2 0 519207691 331378688 6568 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10189/statm: 80903 6568 400 364 0 16809 0
[pid=10191] ppid=10189 vsize=421804 CPUtime=100.13 cores=1,3,5,7
/proc/10191/stat : 10191 (adaptg2wsat2009) T 10189 10191 10191 0 -1 4202496 2139 0 0 0 10013 0 0 0 25 0 1 0 519207692 431927296 1872 33554432000 134512640 135107411 4293309360 18446744073709551615 134531582 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10191/statm: 105451 1872 59 146 0 104327 0
[pid=10189/tid=10292] ppid=10156 vsize=323612 CPUtime=0 cores=1,3,5,7
/proc/10189/task/10292/stat : 10292 (python) S 10156 10156 9843 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519217706 331378688 6568 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604471127115072 0 0 -1 7 0 0 0
[pid=10194] ppid=10156 vsize=333952 CPUtime=0.15 cores=1,3,5,7
/proc/10194/stat : 10194 (python) S 10156 10156 9843 0 -1 4202560 1127 0 0 0 9 6 0 0 19 0 2 0 519207696 341966848 6572 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/10194/statm: 83488 6572 400 364 0 19391 0
[pid=10196] ppid=10194 vsize=1020188 CPUtime=98.36 cores=1,3,5,7
/proc/10196/stat : 10196 (gnovelty+2) T 10194 10196 10196 0 -1 4202496 1141 0 0 0 9835 1 0 0 25 0 1 0 519207696 1044672512 877 33554432000 134512640 135663704 4292939680 18446744073709551615 134544186 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10196/statm: 255047 877 113 282 0 254763 0
[pid=10194/tid=10304] ppid=10156 vsize=333952 CPUtime=0 cores=1,3,5,7
/proc/10194/task/10304/stat : 10304 (python) S 10156 10156 9843 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 519217726 341966848 6572 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604471138065216 0 0 -1 5 0 0 0
[pid=10197] ppid=10156 vsize=344164 CPUtime=0.15 cores=1,3,5,7
/proc/10197/stat : 10197 (python) S 10156 10156 9843 0 -1 4202560 1191 0 0 0 8 7 0 0 19 0 2 0 519207699 352423936 6582 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/10197/statm: 86041 6582 406 364 0 21941 0
[pid=10199] ppid=10197 vsize=251292 CPUtime=99.85 cores=1,3,5,7
/proc/10199/stat : 10199 (FH) T 10197 10199 10199 0 -1 4202496 1960 0 0 0 9985 0 0 0 25 0 1 0 519207699 257323008 1694 33554432000 134512640 134936988 4292392816 18446744073709551615 134545336 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10199/statm: 62823 1694 41 104 0 62327 0
[pid=10197/tid=10296] ppid=10156 vsize=344164 CPUtime=0 cores=1,3,5,7
/proc/10197/task/10296/stat : 10296 (python) S 10156 10156 9843 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 519217716 352423936 6582 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604471127114176 0 0 -1 5 0 0 0
[pid=10200] ppid=10156 vsize=354516 CPUtime=0.15 cores=1,3,5,7
/proc/10200/stat : 10200 (python) S 10156 10156 9843 0 -1 4202560 1173 0 0 0 7 8 0 0 19 0 2 0 519207703 363024384 6586 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10200/statm: 88629 6586 406 364 0 24526 0
[pid=10202] ppid=10200 vsize=421804 CPUtime=100.14 cores=1,3,5,7
/proc/10202/stat : 10202 (adaptg2wsat2009) T 10200 10202 10202 0 -1 4202496 2144 0 0 0 10014 0 0 0 25 0 1 0 519207703 431927296 1871 33554432000 134512640 135107411 4289915840 18446744073709551615 134531246 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10202/statm: 105451 1871 59 146 0 104327 0
[pid=10200/tid=10297] ppid=10156 vsize=354516 CPUtime=0 cores=1,3,5,7
/proc/10200/task/10297/stat : 10297 (python) S 10156 10156 9843 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 519217718 363024384 6586 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604471134303232 0 0 -1 7 0 0 0
[pid=10293] ppid=10156 vsize=364780 CPUtime=0.16 cores=1,3,5,7
/proc/10293/stat : 10293 (python) S 10156 10156 9843 0 -1 4202560 1165 0 0 0 9 7 0 0 19 0 2 0 519217710 373534720 6584 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10293/statm: 91195 6584 400 364 0 27089 0
[pid=10295] ppid=10293 vsize=421804 CPUtime=99 cores=1,3,5,7
/proc/10295/stat : 10295 (adaptg2wsat2009) T 10293 10295 10295 0 -1 4202496 2143 0 0 0 9900 0 0 0 25 0 1 0 519217711 431927296 1872 33554432000 134512640 135107411 4287337760 18446744073709551615 134531613 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10295/statm: 105451 1872 59 146 0 104327 0
[pid=10293/tid=10327] ppid=10156 vsize=364780 CPUtime=0 cores=1,3,5,7
/proc/10293/task/10327/stat : 10327 (python) S 10156 10156 9843 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 519227627 373534720 6584 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604469056651520 0 0 -1 7 0 0 0
[pid=10298] ppid=10156 vsize=375148 CPUtime=0.16 cores=1,3,5,7
/proc/10298/stat : 10298 (python) S 10156 10156 9843 0 -1 4202560 1147 0 0 0 9 7 0 0 19 0 2 0 519217720 384151552 6588 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/10298/statm: 93787 6588 400 364 0 29678 0
[pid=10300] ppid=10298 vsize=251292 CPUtime=100.15 cores=1,3,5,7
/proc/10300/stat : 10300 (FH) T 10298 10300 10300 0 -1 4202496 1965 0 0 0 10015 0 0 0 25 0 1 0 519217721 257323008 1694 33554432000 134512640 134936988 4293304992 18446744073709551615 134552080 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10300/statm: 62823 1694 41 104 0 62327 0
[pid=10298/tid=10331] ppid=10156 vsize=375148 CPUtime=0 cores=1,3,5,7
/proc/10298/task/10331/stat : 10331 (python) S 10156 10156 9843 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519227736 384151552 6588 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604471125181376 0 0 -1 1 0 0 0
[pid=10301] ppid=10156 vsize=385364 CPUtime=0.52 cores=1,3,5,7
/proc/10301/stat : 10301 (python) S 10156 10156 9843 0 -1 4202560 1199 0 0 0 28 24 0 0 18 0 2 0 519217725 394612736 6592 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/10301/statm: 96341 6592 400 364 0 32229 0
[pid=10303] ppid=10301 vsize=268420 CPUtime=291.2 cores=1,3,5,7
/proc/10303/stat : 10303 (hybridGM3) T 10301 10303 10303 0 -1 4202496 5646 0 0 0 29120 0 0 0 25 0 1 0 519217726 274862080 5377 33554432000 134512640 135204385 4288103568 18446744073709551615 134538044 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10303/statm: 67105 5377 63 169 0 66935 0
[pid=10301/tid=10357] ppid=10156 vsize=385364 CPUtime=0 cores=1,3,5,7
/proc/10301/task/10357/stat : 10357 (python) S 10156 10156 9843 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 519237366 394612736 6592 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604461858597568 0 0 -1 1 0 0 0
[pid=10305] ppid=10156 vsize=395748 CPUtime=0.16 cores=1,3,5,7
/proc/10305/stat : 10305 (python) S 10156 10156 9843 0 -1 4202560 1183 0 0 0 8 8 0 0 19 0 2 0 519217730 405245952 6596 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/10305/statm: 98937 6596 400 364 0 34822 0
[pid=10307] ppid=10305 vsize=1020188 CPUtime=98.29 cores=1,3,5,7
/proc/10307/stat : 10307 (gnovelty+2) T 10305 10307 10307 0 -1 4202496 1150 0 0 0 9829 0 0 0 25 0 1 0 519217731 1044672512 876 33554432000 134512640 135663704 4288924704 18446744073709551615 134545631 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10307/statm: 255047 876 113 282 0 254763 0
[pid=10305/tid=10335] ppid=10156 vsize=395748 CPUtime=0 cores=1,3,5,7
/proc/10305/task/10335/stat : 10335 (python) S 10156 10156 9843 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 519227752 405245952 6596 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604461858596672 0 0 -1 5 0 0 0
[pid=10328] ppid=10156 vsize=405924 CPUtime=0.37 cores=1,3,5,7
/proc/10328/stat : 10328 (python) S 10156 10156 9843 0 -1 4202560 1190 0 0 0 20 17 0 0 18 0 2 0 519227632 415666176 6614 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/10328/statm: 101481 6614 400 364 0 37363 0
[pid=10330] ppid=10328 vsize=268428 CPUtime=203.27 cores=1,3,5,7
/proc/10330/stat : 10330 (hybridGM3) R 10328 10330 10330 0 -1 4202496 5652 0 0 0 20327 0 0 0 25 0 1 0 519227632 274870272 5378 33554432000 134512640 135204385 4292994528 18446744073709551615 134538041 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/10330/statm: 67107 5378 63 169 0 66937 0
[pid=10328/tid=10380] ppid=10156 vsize=405924 CPUtime=0 cores=1,3,5,7
/proc/10328/task/10380/stat : 10380 (python) S 10156 10156 9843 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 519247274 415666176 6614 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604467074270528 0 0 -1 5 0 0 0
[pid=10332] ppid=10156 vsize=416320 CPUtime=0.18 cores=1,3,5,7
/proc/10332/stat : 10332 (python) S 10156 10156 9843 0 -1 4202560 1185 0 0 0 9 9 0 0 19 0 2 0 519227741 426311680 6618 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/10332/statm: 104080 6618 400 364 0 39959 0
[pid=10334] ppid=10332 vsize=421804 CPUtime=99.65 cores=1,3,5,7
/proc/10334/stat : 10334 (adaptg2wsat2009) T 10332 10334 10334 0 -1 4202496 2146 0 0 0 9965 0 0 0 25 0 1 0 519227741 431927296 1872 33554432000 134512640 135107411 4289355424 18446744073709551615 134531239 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10334/statm: 105451 1872 59 146 0 104327 0
[pid=10332/tid=10359] ppid=10156 vsize=416320 CPUtime=0 cores=1,3,5,7
/proc/10332/task/10359/stat : 10359 (python) S 10156 10156 9843 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 519237759 426311680 6618 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604467074271424 0 0 -1 3 0 0 0
[pid=10336] ppid=10156 vsize=426876 CPUtime=0.36 cores=1,3,5,7
/proc/10336/stat : 10336 (python) S 10156 10156 9843 0 -1 4202560 1234 0 0 0 21 15 0 0 19 0 2 0 519227757 437121024 6645 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/10336/statm: 106719 6645 408 364 0 42595 0
[pid=10338] ppid=10336 vsize=268424 CPUtime=197.65 cores=1,3,5,7
/proc/10338/stat : 10338 (hybridGM3) T 10336 10338 10338 0 -1 4202496 5653 0 0 0 19764 1 0 0 25 0 1 0 519227757 274866176 5378 33554432000 134512640 135204385 4290238944 18446744073709551615 134538284 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/10338/statm: 67106 5378 63 169 0 66936 0
[pid=10336/tid=10381] ppid=10156 vsize=426876 CPUtime=0 cores=1,3,5,7
/proc/10336/task/10381/stat : 10381 (python) S 10156 10156 9843 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 519248000 437121024 6645 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 514 18446604463851170752 0 0 -1 5 0 0 0
[pid=10360] ppid=10156 vsize=436856 CPUtime=0.18 cores=1,3,5,7
/proc/10360/stat : 10360 (python) S 10156 10156 9843 0 -1 4202560 1082 0 0 0 10 8 0 0 18 0 1 0 519237764 447340544 6695 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 16935645661238362374 0 0 17 5 0 0 0
/proc/10360/statm: 109214 6695 380 364 0 45087 0
[pid=10362] ppid=10360 vsize=268424 CPUtime=101.94 cores=1,3,5,7
/proc/10362/stat : 10362 (hybridGM3) R 10360 10362 10362 0 -1 4202496 5655 0 0 0 10194 0 0 0 25 0 1 0 519237765 274866176 5377 33554432000 134512640 135204385 4292259696 18446744073709551615 134537797 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10362/statm: 67106 5377 63 169 0 66936 0
[pid=10377] ppid=10156 vsize=447276 CPUtime=0.02 cores=1,3,5,7
/proc/10377/stat : 10377 (python) S 10156 10156 9843 0 -1 4202560 1024 0 0 0 1 1 0 0 18 0 1 0 519246892 458010624 6693 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 16935645661238362376 0 0 17 5 0 0 0
/proc/10377/statm: 111819 6693 374 364 0 47689 0
[pid=10379] ppid=10377 vsize=459768 CPUtime=11.14 cores=1,3,5,7
/proc/10379/stat : 10379 (TNM) R 10377 10379 10379 0 -1 4202496 1930 0 0 0 1114 0 0 0 25 0 1 0 519246892 470802432 1657 33554432000 134512640 135008748 4292933376 18446744073709551615 134527503 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/10379/statm: 114942 1657 46 122 0 113842 0
[pid=10382] ppid=10156 vsize=457712 CPUtime=0 cores=1,3,5,7
/proc/10382/stat : 10382 (python) S 10156 10156 9843 0 -1 4202560 839 0 0 0 0 0 0 0 19 0 1 0 519248005 468697088 6669 33554432000 4194304 5683644 140734217194144 18446744073709551615 212039675462 0 0 16781312 514 4435068446295719935 0 0 17 5 0 0 0
/proc/10382/statm: 114428 6669 346 364 0 50295 0
[pid=10384] ppid=10382 vsize=245848 CPUtime=0.04 cores=1,3,5,7
/proc/10384/stat : 10384 (FH) R 10382 10384 10384 0 -1 4202496 708 0 0 0 4 0 0 0 20 0 1 0 519248005 251748352 436 33554432000 134512640 134936988 4293173776 18446744073709551615 134521553 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10384/statm: 61462 437 37 104 0 60966 0
[pid=10156/tid=10169] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10169/stat : 10169 (python) S 10154 10156 9843 0 -1 4202560 26 935 0 0 0 0 0 0 24 0 19 0 519197669 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 1 0 0 0
[pid=10156/tid=10172] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10172/stat : 10172 (python) S 10154 10156 9843 0 -1 4202560 24 935 0 0 0 0 0 0 25 0 19 0 519197672 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10175] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10175/stat : 10175 (python) S 10154 10156 9843 0 -1 4202560 23 935 0 0 0 0 0 0 20 0 19 0 519197675 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 1 0 0 0
[pid=10156/tid=10178] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10178/stat : 10178 (python) S 10154 10156 9843 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 19 0 519197678 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 7 0 0 0
[pid=10156/tid=10190] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10190/stat : 10190 (python) S 10154 10156 9843 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 519207691 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 7 0 0 0
[pid=10156/tid=10195] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10195/stat : 10195 (python) S 10154 10156 9843 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 19 0 519207696 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 1 0 0 0
[pid=10156/tid=10198] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10198/stat : 10198 (python) S 10154 10156 9843 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 19 0 519207699 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10201] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10201/stat : 10201 (python) S 10154 10156 9843 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 19 0 519207703 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 7 0 0 0
[pid=10156/tid=10294] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10294/stat : 10294 (python) S 10154 10156 9843 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 19 0 519217710 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 7 0 0 0
[pid=10156/tid=10299] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10299/stat : 10299 (python) S 10154 10156 9843 0 -1 4202560 25 935 0 0 0 0 0 0 21 0 19 0 519217721 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 1 0 0 0
[pid=10156/tid=10302] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10302/stat : 10302 (python) S 10154 10156 9843 0 -1 4202560 40 935 0 0 0 0 0 0 18 0 19 0 519217725 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10306] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10306/stat : 10306 (python) S 10154 10156 9843 0 -1 4202560 24 935 0 0 0 0 0 0 21 0 19 0 519217731 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10329] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10329/stat : 10329 (python) S 10154 10156 9843 0 -1 4202560 34 935 0 0 0 0 0 0 18 0 19 0 519227632 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10333] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10333/stat : 10333 (python) S 10154 10156 9843 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 19 0 519227741 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10337] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10337/stat : 10337 (python) S 10154 10156 9843 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 519227757 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10361] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10361/stat : 10361 (python) S 10154 10156 9843 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 519237764 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10378] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10378/stat : 10378 (python) S 10154 10156 9843 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 519246892 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
[pid=10156/tid=10383] ppid=10154 vsize=467404 CPUtime=0 cores=1,3,5,7
/proc/10156/task/10383/stat : 10383 (python) S 10154 10156 9843 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 19 0 519248005 478621696 8238 33554432000 4194304 5683644 140734217194144 18446744073709551615 212051478961 0 0 16781312 2 18446604471145673920 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2006.63
Current children cumulated vsize (KiB) 16030808

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 10156 and gives
#  childrusage.ru_utime.tv_sec=3
#  childrusage.ru_utime.tv_usec=70533
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=142978
# CPU time returned by wait4() is 3.21351
# while last known CPU time is 2006.63
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 506.022
CPU time (s): 2006.63
CPU user time (s): 2004.93
CPU system time (s): 1.7
CPU usage (%): 396.55
Max. virtual memory (cumulated for all children) (KiB): 16030808

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3.07053
system time used= 0.142978
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 31752
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1304
involuntary context switches= 89

runsolver used 2.6306 second user time and 6.26705 second system time

The end

Launcher Data 

Begin job on node146 at 2011-05-01 18:36:51
IDJOB=3317155
IDBENCH=82620
IDSOLVER=1705
FILE ID=node146/3317155-1304267811
RUNJOBID= node146-1304267279-9862
PBS_JOBID= 13173933
Free space on /tmp= 73320 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k7-r85-v400-c34000-S943787219-031-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317155-1304267811/watcher-3317155-1304267811 -o /tmp/evaluation-result-3317155-1304267811/solver-3317155-1304267811 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317155-1304267811.cnf -seed 1317240943 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 6105ec111e1e16da586afadcc6f09041
RANDOM SEED=1317240943

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30478744 kB
Buffers:        461904 kB
Cached:        1296500 kB
SwapCached:        888 kB
Active:         568660 kB
Inactive:      1517408 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30478744 kB
SwapTotal:    67111528 kB
SwapFree:     67106388 kB
Dirty:           67500 kB
Writeback:          40 kB
AnonPages:      326912 kB
Mapped:          22384 kB
Slab:           317792 kB
PageTables:      10636 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 16192724 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73292 MiB
End job on node146 at 2011-05-01 18:45:20