Trace number 3378495

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? 4782.76 1199.98

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v650-c2769-S910125393-002.UNKNOWN.cnf
MD5SUM8567972e0c3ca849f27342fc5fce028c
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables650
Number of clauses2769
Sum of the clauses size8307
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32769
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.35/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.67	c INFO: solving HOME/instance-3378495-1305465929.cnf
0.35/0.67	c INFO: computing task features
0.35/0.68	c INFO: computed variable-clause graph statistics
0.35/0.68	c INFO: computed clause balance statistics
0.42/0.73	c INFO: computed variable balance statistics
0.42/0.77	c INFO: computed small-clause counts
0.42/0.77	c INFO: computed Horn-clause counts
0.42/0.77	c INFO: computed variable graph statistics
0.42/0.78	c INFO: collected features for HOME/instance-3378495-1305465929.cnf in 0.10 s
0.42/0.78	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.58)
0.42/0.79	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.29)
0.51/0.80	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.07)
0.51/0.81	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.27)
392.19/99.04	c INFO: running march_hi@0 for 679 with 1101 remaining (b = 0.34)
400.11/101.06	c INFO: running march_hi@0 for 679 with 1099 remaining (b = 0.30)
1559.54/391.79	c INFO: running march_hi@0 for 291 with 808 remaining (b = 0.16)
1561.55/392.20	c INFO: running march_hi@0 for 582 with 808 remaining (b = 0.25)
2723.32/683.58	c INFO: running march_hi@389 for 97 with 517 remaining (b = 0.06)
3112.95/781.21	c INFO: running march_hi@98 for 291 with 419 remaining (b = 0.13)
3129.34/785.34	c INFO: running march_hi@0 for 415 with 415 remaining (b = 0.17)
3157.60/792.47	c INFO: running march_hi@98 for 291 with 408 remaining (b = 0.11)
3904.07/979.68	c INFO: running march_hi@290 for 220 with 220 remaining (b = 0.09)
4288.00/1075.93	c INFO: running march_hi@390 for 124 with 124 remaining (b = 0.05)
4317.49/1083.40	c INFO: running march_hi@487 for 117 with 117 remaining (b = 0.04)
4782.35/1199.96	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378495-1305465929/watcher-3378495-1305465929 -o /tmp/evaluation-result-3378495-1305465929/solver-3378495-1305465929 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378495-1305465929.cnf -seed 736253521 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.37 9.27 8.78 7/368 32245
/proc/meminfo: memFree=10586732/32951124 swapFree=66050672/67111528
[pid=32245] ppid=32243 vsize=15772 CPUtime=0 cores=1,3,5,7
/proc/32245/stat : 32245 (python) R 32243 32245 30970 0 -1 4202496 902 0 0 0 0 0 0 0 25 0 1 0 658190776 16150528 783 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377299456 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/32245/statm: 3943 783 346 364 0 430 0

[startup+0.0339609 s]
/proc/loadavg: 9.37 9.27 8.78 7/368 32245
/proc/meminfo: memFree=10586732/32951124 swapFree=66050672/67111528
[pid=32245] ppid=32243 vsize=16808 CPUtime=0.01 cores=1,3,5,7
/proc/32245/stat : 32245 (python) S 32243 32245 30970 0 -1 4202496 1212 0 0 0 1 0 0 0 25 0 1 0 658190776 17211392 1071 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377299456 0 2147483391 16781312 2 18446744071699453118 0 0 17 5 0 0 0
/proc/32245/statm: 4202 1071 417 364 0 689 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16808

[startup+0.100962 s]
/proc/loadavg: 9.37 9.27 8.78 7/368 32245
/proc/meminfo: memFree=10586732/32951124 swapFree=66050672/67111528
[pid=32245] ppid=32243 vsize=87700 CPUtime=0.06 cores=1,3,5,7
/proc/32245/stat : 32245 (python) S 32243 32245 30970 0 -1 4202496 2637 0 0 0 5 1 0 0 17 0 1 0 658190776 89804800 2353 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377299456 0 2147483391 16781312 2 18446744071699453118 0 0 17 5 0 0 0
/proc/32245/statm: 21925 2353 717 364 0 1621 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 87700

[startup+0.300957 s]
/proc/loadavg: 9.37 9.27 8.78 7/368 32245
/proc/meminfo: memFree=10586732/32951124 swapFree=66050672/67111528
[pid=32245] ppid=32243 vsize=135748 CPUtime=0.18 cores=1,3,5,7
/proc/32245/stat : 32245 (python) S 32243 32245 30970 0 -1 4202496 5962 933 1 0 13 5 0 0 17 0 1 0 658190776 139005952 4150 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377297413 0 2147483391 16781312 2 18446744071699453118 0 0 17 5 0 0 0
/proc/32245/statm: 33937 4150 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 135748

[startup+0.700822 s]
/proc/loadavg: 9.37 9.27 8.78 7/368 32245
/proc/meminfo: memFree=10586732/32951124 swapFree=66050672/67111528
[pid=32245] ppid=32243 vsize=278548 CPUtime=0.42 cores=1,3,5,7
/proc/32245/stat : 32245 (python) R 32243 32245 30970 0 -1 4202496 9357 933 1 0 31 11 0 0 15 0 1 0 658190776 285233152 7028 33554432000 4194304 5683644 140737049898512 18446744073709551615 5005930 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/32245/statm: 69637 7028 1826 364 0 5561 0
Current children cumulated CPU time (s) 0.42
Current children cumulated vsize (KiB) 278548

[startup+1.50064 s]
/proc/loadavg: 9.37 9.27 8.78 10/381 32262
/proc/meminfo: memFree=10546200/32951124 swapFree=66050672/67111528
[pid=32245] ppid=32243 vsize=320276 CPUtime=0.53 cores=1,3,5,7
/proc/32245/stat : 32245 (python) S 32243 32245 30970 0 -1 4202496 12209 933 1 0 42 11 0 0 24 0 5 0 658190776 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/32245/statm: 80069 7266 1881 364 0 15978 0
[pid=32251] ppid=32245 vsize=278572 CPUtime=0 cores=1,3,5,7
/proc/32251/stat : 32251 (python) S 32245 32245 30970 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 658190855 285257728 5571 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321487 0 0 16781312 514 11513351296597884927 0 0 17 7 0 0 0
/proc/32251/statm: 69643 5571 354 364 0 5561 0
[pid=32253] ppid=32251 vsize=1017940 CPUtime=0.7 cores=1,3,5,7
/proc/32253/stat : 32253 (gnovelty+2) R 32251 32253 32253 0 -1 4202496 456 0 0 0 70 0 0 0 21 0 1 0 658190855 1042370560 192 33554432000 134512640 135663704 4287541184 18446744073709551615 134545598 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/32253/statm: 254485 192 113 282 0 254201 0
[pid=32254] ppid=32245 vsize=289416 CPUtime=0 cores=1,3,5,7
/proc/32254/stat : 32254 (python) S 32245 32245 30970 0 -1 4202560 748 0 0 0 0 0 0 0 15 0 1 0 658190856 296361984 5712 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 5618183364834984198 0 0 17 5 0 0 0
/proc/32254/statm: 72354 5712 348 364 0 8269 0
[pid=32256] ppid=32254 vsize=2796 CPUtime=0.69 cores=1,3,5,7
/proc/32256/stat : 32256 (march_hi) R 32254 32256 32256 0 -1 4202496 741 0 0 0 69 0 0 0 22 0 1 0 658190856 2863104 478 33554432000 4194304 4955518 140734524727856 18446744073709551615 4289745 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/32256/statm: 699 478 104 186 0 512 0
[pid=32257] ppid=32245 vsize=299168 CPUtime=0 cores=1,3,5,7
/proc/32257/stat : 32257 (python) S 32245 32245 30970 0 -1 4202560 735 0 0 0 0 0 0 0 21 0 1 0 658190857 306348032 5605 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 2031849388791300095 0 0 17 7 0 0 0
/proc/32257/statm: 74792 5605 348 364 0 10704 0
[pid=32259] ppid=32257 vsize=1017940 CPUtime=0.67 cores=1,3,5,7
/proc/32259/stat : 32259 (gnovelty+2) R 32257 32259 32259 0 -1 4202496 450 0 0 0 67 0 0 0 25 0 1 0 658190857 1042370560 192 33554432000 134512640 135663704 4289339600 18446744073709551615 134524594 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/32259/statm: 254485 192 113 282 0 254201 0
[pid=32260] ppid=32245 vsize=310032 CPUtime=0 cores=1,3,5,7
/proc/32260/stat : 32260 (python) S 32245 32245 30970 0 -1 4202560 753 0 0 0 0 0 0 0 18 0 1 0 658190858 317472768 5729 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 12671141438692491526 0 0 17 5 0 0 0
/proc/32260/statm: 77508 5729 348 364 0 13417 0
[pid=32262] ppid=32260 vsize=2796 CPUtime=0.67 cores=1,3,5,7
/proc/32262/stat : 32262 (march_hi) R 32260 32262 32262 0 -1 4202496 738 0 0 0 67 0 0 0 25 0 1 0 658190858 2863104 477 33554432000 4194304 4955518 140736794816432 18446744073709551615 4238469 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32262/statm: 699 477 104 186 0 512 0
[pid=32245/tid=32252] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32252/stat : 32252 (python) S 32243 32245 30970 0 -1 4202560 26 933 0 0 0 0 0 0 18 0 5 0 658190855 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 7 0 0 0
[pid=32245/tid=32255] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32255/stat : 32255 (python) S 32243 32245 30970 0 -1 4202560 22 933 0 0 0 0 0 0 20 0 5 0 658190856 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 3 0 0 0
[pid=32245/tid=32258] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32258/stat : 32258 (python) S 32243 32245 30970 0 -1 4202560 24 933 0 0 0 0 0 0 22 0 5 0 658190857 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 7 0 0 0
[pid=32245/tid=32261] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32261/stat : 32261 (python) S 32243 32245 30970 0 -1 4202560 28 933 0 0 0 0 0 0 24 0 5 0 658190858 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3.26
Current children cumulated vsize (KiB) 3538936

[startup+3.10134 s]
/proc/loadavg: 9.37 9.27 8.78 7/357 32293
/proc/meminfo: memFree=11190372/32951124 swapFree=66050672/67111528
[pid=32245] ppid=32243 vsize=320276 CPUtime=0.53 cores=1,3,5,7
/proc/32245/stat : 32245 (python) S 32243 32245 30970 0 -1 4202496 12209 933 1 0 42 11 0 0 24 0 5 0 658190776 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/32245/statm: 80069 7266 1881 364 0 15978 0
[pid=32251] ppid=32245 vsize=278572 CPUtime=0 cores=1,3,5,7
/proc/32251/stat : 32251 (python) S 32245 32245 30970 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 658190855 285257728 5571 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321487 0 0 16781312 514 11513351296597852423 0 0 17 7 0 0 0
/proc/32251/statm: 69643 5571 354 364 0 5561 0
[pid=32253] ppid=32251 vsize=1017940 CPUtime=2.3 cores=1,3,5,7
/proc/32253/stat : 32253 (gnovelty+2) R 32251 32253 32253 0 -1 4202496 456 0 0 0 230 0 0 0 25 0 1 0 658190855 1042370560 192 33554432000 134512640 135663704 4287541184 18446744073709551615 134547247 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/32253/statm: 254485 192 113 282 0 254201 0
[pid=32254] ppid=32245 vsize=289416 CPUtime=0 cores=1,3,5,7
/proc/32254/stat : 32254 (python) S 32245 32245 30970 0 -1 4202560 750 0 0 0 0 0 0 0 15 0 1 0 658190856 296361984 5712 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 5618183364834984198 0 0 17 5 0 0 0
/proc/32254/statm: 72354 5712 348 364 0 8269 0
[pid=32256] ppid=32254 vsize=3116 CPUtime=2.29 cores=1,3,5,7
/proc/32256/stat : 32256 (march_hi) R 32254 32256 32256 0 -1 4202496 813 0 0 0 229 0 0 0 25 0 1 0 658190856 3190784 550 33554432000 4194304 4955518 140734524727856 18446744073709551615 4224668 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/32256/statm: 779 550 104 186 0 592 0
[pid=32257] ppid=32245 vsize=299168 CPUtime=0 cores=1,3,5,7
/proc/32257/stat : 32257 (python) S 32245 32245 30970 0 -1 4202560 735 0 0 0 0 0 0 0 18 0 1 0 658190857 306348032 5605 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 2031849388791267591 0 0 17 7 0 0 0
/proc/32257/statm: 74792 5605 348 364 0 10704 0
[pid=32259] ppid=32257 vsize=1017940 CPUtime=2.26 cores=1,3,5,7
/proc/32259/stat : 32259 (gnovelty+2) R 32257 32259 32259 0 -1 4202496 450 0 0 0 226 0 0 0 25 0 1 0 658190857 1042370560 192 33554432000 134512640 135663704 4289339600 18446744073709551615 134547286 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/32259/statm: 254485 192 113 282 0 254201 0
[pid=32260] ppid=32245 vsize=310032 CPUtime=0 cores=1,3,5,7
/proc/32260/stat : 32260 (python) S 32245 32245 30970 0 -1 4202560 753 0 0 0 0 0 0 0 15 0 1 0 658190858 317472768 5729 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 12671141438692491526 0 0 17 5 0 0 0
/proc/32260/statm: 77508 5729 348 364 0 13417 0
[pid=32262] ppid=32260 vsize=3116 CPUtime=2.27 cores=1,3,5,7
/proc/32262/stat : 32262 (march_hi) R 32260 32262 32262 0 -1 4202496 810 0 0 0 227 0 0 0 25 0 1 0 658190858 3190784 549 33554432000 4194304 4955518 140736794816432 18446744073709551615 4227583 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32262/statm: 779 549 104 186 0 592 0
[pid=32245/tid=32252] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32252/stat : 32252 (python) S 32243 32245 30970 0 -1 4202560 26 933 0 0 0 0 0 0 18 0 5 0 658190855 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 7 0 0 0
[pid=32245/tid=32255] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32255/stat : 32255 (python) S 32243 32245 30970 0 -1 4202560 22 933 0 0 0 0 0 0 20 0 5 0 658190856 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 3 0 0 0
[pid=32245/tid=32258] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32258/stat : 32258 (python) S 32243 32245 30970 0 -1 4202560 24 933 0 0 0 0 0 0 22 0 5 0 658190857 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 7 0 0 0
[pid=32245/tid=32261] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32261/stat : 32261 (python) S 32243 32245 30970 0 -1 4202560 28 933 0 0 0 0 0 0 24 0 5 0 658190858 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 9.65
Current children cumulated vsize (KiB) 3539576

[startup+6.30075 s]
/proc/loadavg: 9.34 9.26 8.78 10/369 32305
/proc/meminfo: memFree=11161752/32951124 swapFree=66050672/67111528
[pid=32245] ppid=32243 vsize=320276 CPUtime=0.53 cores=1,3,5,7
/proc/32245/stat : 32245 (python) S 32243 32245 30970 0 -1 4202496 12209 933 1 0 42 11 0 0 24 0 5 0 658190776 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/32245/statm: 80069 7266 1881 364 0 15978 0
[pid=32251] ppid=32245 vsize=278572 CPUtime=0 cores=1,3,5,7
/proc/32251/stat : 32251 (python) S 32245 32245 30970 0 -1 4202560 894 0 0 0 0 0 0 0 18 0 1 0 658190855 285257728 5594 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321487 0 0 16781312 514 11513351296597852423 0 0 17 7 0 0 0
/proc/32251/statm: 69643 5594 377 364 0 5561 0
[pid=32253] ppid=32251 vsize=1017940 CPUtime=5.5 cores=1,3,5,7
/proc/32253/stat : 32253 (gnovelty+2) R 32251 32253 32253 0 -1 4202496 456 0 0 0 550 0 0 0 25 0 1 0 658190855 1042370560 192 33554432000 134512640 135663704 4287541184 18446744073709551615 134544264 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/32253/statm: 254485 192 113 282 0 254201 0
[pid=32254] ppid=32245 vsize=289416 CPUtime=0 cores=1,3,5,7
/proc/32254/stat : 32254 (python) S 32245 32245 30970 0 -1 4202560 907 0 0 0 0 0 0 0 15 0 1 0 658190856 296361984 5734 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 5618183364834984198 0 0 17 5 0 0 0
/proc/32254/statm: 72354 5734 370 364 0 8269 0
[pid=32256] ppid=32254 vsize=3832 CPUtime=5.48 cores=1,3,5,7
/proc/32256/stat : 32256 (march_hi) R 32254 32256 32256 0 -1 4202496 959 0 0 0 548 0 0 0 25 0 1 0 658190856 3923968 696 33554432000 4194304 4955518 140734524727856 18446744073709551615 4227807 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/32256/statm: 958 696 104 186 0 771 0
[pid=32257] ppid=32245 vsize=299168 CPUtime=0 cores=1,3,5,7
/proc/32257/stat : 32257 (python) S 32245 32245 30970 0 -1 4202560 889 0 0 0 0 0 0 0 18 0 1 0 658190857 306348032 5628 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 2031849388791267591 0 0 17 7 0 0 0
/proc/32257/statm: 74792 5628 371 364 0 10704 0
[pid=32259] ppid=32257 vsize=1017940 CPUtime=5.39 cores=1,3,5,7
/proc/32259/stat : 32259 (gnovelty+2) R 32257 32259 32259 0 -1 4202496 450 0 0 0 539 0 0 0 25 0 1 0 658190857 1042370560 192 33554432000 134512640 135663704 4289339600 18446744073709551615 134544308 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/32259/statm: 254485 192 113 282 0 254201 0
[pid=32260] ppid=32245 vsize=310032 CPUtime=0.01 cores=1,3,5,7
/proc/32260/stat : 32260 (python) S 32245 32245 30970 0 -1 4202560 910 0 0 0 0 1 0 0 15 0 1 0 658190858 317472768 5752 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 12671141438692491526 0 0 17 5 0 0 0
/proc/32260/statm: 77508 5752 371 364 0 13417 0
[pid=32262] ppid=32260 vsize=3832 CPUtime=5.44 cores=1,3,5,7
/proc/32262/stat : 32262 (march_hi) R 32260 32262 32262 0 -1 4202496 956 0 0 0 544 0 0 0 25 0 1 0 658190858 3923968 695 33554432000 4194304 4955518 140736794816432 18446744073709551615 4227741 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32262/statm: 958 695 104 186 0 771 0
[pid=32245/tid=32252] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32252/stat : 32252 (python) S 32243 32245 30970 0 -1 4202560 26 933 0 0 0 0 0 0 18 0 5 0 658190855 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 7 0 0 0
[pid=32245/tid=32255] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32255/stat : 32255 (python) S 32243 32245 30970 0 -1 4202560 22 933 0 0 0 0 0 0 20 0 5 0 658190856 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 3 0 0 0
[pid=32245/tid=32258] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32258/stat : 32258 (python) S 32243 32245 30970 0 -1 4202560 24 933 0 0 0 0 0 0 22 0 5 0 658190857 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 7 0 0 0
[pid=32245/tid=32261] ppid=32243 vsize=320276 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32261/stat : 32261 (python) S 32243 32245 30970 0 -1 4202560 28 933 0 0 0 0 0 0 24 0 5 0 658190858 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 22.35
Current children cumulated vsize (KiB) 3541008

[startup+12.7006 s]
/proc/loadavg: 9.40 9.27 8.78 11/369 32306
/proc/meminfo: memFree=11158216/32951124 swapFree=66050672/67111528
[pid=32245] ppid=32243 vsize=320276 CPUtime=0.53 cores=1,3,5,7
/proc/32245/stat : 32245 (python) S 32243 32245 30970 0 -1 4202496 12209 933 1 0 42 11 0 0 24 0 5 0 658190776 327962624 7266 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/32245/statm: 80069 7266 1881 364 0 15978 0
[pid=32251] ppid=32245 vsize=278572 CPUtime=0.02 cores=1,3,5,7
/proc/32251/stat : 32251 (python) S 32245 32245 30970 0 -1 4202560 898 0 0 0 1 1 0 0 18 0 1 0 658190855 285257728 5594 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321487 0 0 16781312 514 9092679686730776839 0 0 17 7 0 0 0
/proc/32251/statm: 69643 5594 377 364 0 5561 0
[pid=32253] ppid=32251 vsize=1017940 CPUtime=11.9 cores=1,3,5,7
/proc/32253/stat : 32253 (gnovelty+2) R 32251 32253 32253 0 -1 4202496 456 0 0 0 1190 0 0 0 25 0 1 0 658190855 1042370560 192 33554432000 134512640 135663704 4287541184 18446744073709551615 134547041 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/32253/statm: 254485 192 113 282 0 254201 0
[pid=32254] ppid=32245 vsize=289416 CPUtime=0.02 cores=1,3,5,7

################
# More data... #
################

[pid=32245] ppid=32243 vsize=360988 CPUtime=0.61 cores=1,3,5,7
/proc/32245/stat : 32245 (python) S 32243 32245 30970 0 -1 4202496 16347 933 1 0 49 12 0 0 18 0 9 0 658190776 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/32245/statm: 90247 7245 1890 364 0 26144 0
[pid=32251] ppid=32245 vsize=288816 CPUtime=0.25 cores=1,3,5,7
/proc/32251/stat : 32251 (python) S 32245 32245 30970 0 -1 4202560 1165 0 0 0 13 12 0 0 19 0 2 0 658190855 295747584 5638 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/32251/statm: 72204 5638 416 364 0 8122 0
[pid=32253] ppid=32251 vsize=1017940 CPUtime=98.24 cores=1,3,5,7
/proc/32253/stat : 32253 (gnovelty+2) T 32251 32253 32253 0 -1 4202496 456 0 0 0 9824 0 0 0 25 0 1 0 658190855 1042370560 192 33554432000 134512640 135663704 4287541184 18446744073709551615 134545644 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/32253/statm: 254485 192 113 282 0 254201 0
[pid=32251/tid=32309] ppid=32245 vsize=288816 CPUtime=0 cores=1,3,5,7
/proc/32251/task/32309/stat : 32309 (python) S 32245 32245 30970 0 -1 4202560 42 0 0 0 0 0 0 0 19 0 2 0 658200680 295747584 5638 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604464183288832 0 0 -1 7 0 0 0
[pid=32254] ppid=32245 vsize=289416 CPUtime=2.44 cores=1,3,5,7
/proc/32254/stat : 32254 (python) S 32245 32245 30970 0 -1 4202560 1146 0 0 0 128 116 0 0 15 0 2 0 658190856 296361984 5765 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/32254/statm: 72354 5765 400 364 0 8269 0
[pid=32256] ppid=32254 vsize=173568 CPUtime=894.21 cores=1,3,5,7
/proc/32256/stat : 32256 (march_hi) R 32254 32256 32256 0 -1 4202496 31144 0 0 0 89416 5 0 0 25 0 1 0 658190856 177733632 30881 33554432000 4194304 4955518 140734524727856 18446744073709551615 4227647 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32256/statm: 43392 30881 104 186 0 43205 0
[pid=32254/tid=32333] ppid=32245 vsize=289416 CPUtime=0 cores=1,3,5,7
/proc/32254/task/32333/stat : 32333 (python) S 32245 32245 30970 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 658229955 296361984 5765 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604470122893376 0 0 -1 3 0 0 0
[pid=32257] ppid=32245 vsize=299168 CPUtime=0.25 cores=1,3,5,7
/proc/32257/stat : 32257 (python) S 32245 32245 30970 0 -1 4202560 1077 0 0 0 16 9 0 0 19 0 2 0 658190857 306348032 5658 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/32257/statm: 74792 5658 400 364 0 10704 0
[pid=32259] ppid=32257 vsize=1017940 CPUtime=98.66 cores=1,3,5,7
/proc/32259/stat : 32259 (gnovelty+2) T 32257 32259 32259 0 -1 4202496 450 0 0 0 9866 0 0 0 25 0 1 0 658190857 1042370560 192 33554432000 134512640 135663704 4289339600 18446744073709551615 134524498 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/32259/statm: 254485 192 113 282 0 254201 0
[pid=32257/tid=32313] ppid=32245 vsize=299168 CPUtime=0 cores=1,3,5,7
/proc/32257/task/32313/stat : 32313 (python) S 32245 32245 30970 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 658200881 306348032 5658 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604457680737344 0 0 -1 7 0 0 0
[pid=32260] ppid=32245 vsize=310032 CPUtime=1.87 cores=1,3,5,7
/proc/32260/stat : 32260 (python) S 32245 32245 30970 0 -1 4202560 1159 0 0 0 100 87 0 0 15 0 2 0 658190858 317472768 5782 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/32260/statm: 77508 5782 400 364 0 13417 0
[pid=32262] ppid=32260 vsize=110412 CPUtime=684.55 cores=1,3,5,7
/proc/32262/stat : 32262 (march_hi) T 32260 32262 32262 0 -1 4202496 24276 0 0 0 68450 5 0 0 25 0 1 0 658190858 113061888 24015 33554432000 4194304 4955518 140736794816432 18446744073709551615 4224512 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/32262/statm: 27603 24015 104 186 0 27416 0
[pid=32260/tid=32334] ppid=32245 vsize=310032 CPUtime=0 cores=1,3,5,7
/proc/32260/task/32334/stat : 32334 (python) S 32245 32245 30970 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 658229996 317472768 5782 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604464367268992 0 0 -1 5 0 0 0
[pid=32310] ppid=32245 vsize=320288 CPUtime=2.72 cores=1,3,5,7
/proc/32310/stat : 32310 (python) S 32245 32245 30970 0 -1 4202560 1149 0 0 0 143 129 0 0 15 0 2 0 658200681 327974912 5786 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/32310/statm: 80072 5786 400 364 0 15978 0
[pid=32312] ppid=32310 vsize=173568 CPUtime=972.27 cores=1,3,5,7
/proc/32312/stat : 32312 (march_hi) T 32310 32312 32312 0 -1 4202496 33424 0 0 0 97220 7 0 0 25 0 1 0 658200681 177733632 33163 33554432000 4194304 4955518 140735196371296 18446744073709551615 4314282 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/32312/statm: 43392 33163 104 186 0 43205 0
[pid=32310/tid=32356] ppid=32245 vsize=320288 CPUtime=0 cores=1,3,5,7
/proc/32310/task/32356/stat : 32356 (python) S 32245 32245 30970 0 -1 4202560 45 0 0 0 0 0 0 0 15 0 2 0 658270022 327974912 5786 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604469906150336 0 0 -1 7 0 0 0
[pid=32314] ppid=32245 vsize=330016 CPUtime=2.2 cores=1,3,5,7
/proc/32314/stat : 32314 (python) S 32245 32245 30970 0 -1 4202560 1145 0 0 0 113 107 0 0 15 0 2 0 658200882 337936384 5690 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/32314/statm: 82504 5690 400 364 0 18407 0
[pid=32316] ppid=32314 vsize=129368 CPUtime=805.6 cores=1,3,5,7
/proc/32316/stat : 32316 (march_hi) R 32314 32316 32316 0 -1 4202496 28314 0 0 0 80555 5 0 0 25 0 1 0 658200883 132472832 28057 33554432000 4194304 4955518 140734991633520 18446744073709551615 4284882 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/32316/statm: 32342 28057 104 186 0 32155 0
[pid=32314/tid=32352] ppid=32245 vsize=330016 CPUtime=0 cores=1,3,5,7
/proc/32314/task/32352/stat : 32352 (python) S 32245 32245 30970 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 658269309 337936384 5690 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604469906151232 0 0 -1 1 0 0 0
[pid=32335] ppid=32245 vsize=340896 CPUtime=2.21 cores=1,3,5,7
/proc/32335/stat : 32335 (python) S 32245 32245 30970 0 -1 4202560 1164 0 0 0 117 104 0 0 15 0 2 0 658229997 349077504 5819 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/32335/statm: 85224 5819 400 364 0 21124 0
[pid=32337] ppid=32335 vsize=129368 CPUtime=805.04 cores=1,3,5,7
/proc/32337/stat : 32337 (march_hi) R 32335 32337 32337 0 -1 4202496 28270 0 0 0 80497 7 0 0 25 0 1 0 658229997 132472832 28008 33554432000 4194304 4955518 140736798769136 18446744073709551615 4227671 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/32337/statm: 32342 28008 104 186 0 32155 0
[pid=32335/tid=32367] ppid=32245 vsize=340896 CPUtime=0 cores=1,3,5,7
/proc/32335/task/32367/stat : 32367 (python) S 32245 32245 30970 0 -1 4202560 49 0 0 0 0 0 0 0 17 0 2 0 658288744 349077504 5819 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604468630841280 0 0 -1 1 0 0 0
[pid=32353] ppid=32245 vsize=351352 CPUtime=1.16 cores=1,3,5,7
/proc/32353/stat : 32353 (python) S 32245 32245 30970 0 -1 4202560 1043 0 0 0 61 55 0 0 15 0 1 0 658269310 359784448 5848 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 9092679686730776839 0 0 17 5 0 0 0
/proc/32353/statm: 87838 5848 377 364 0 23735 0
[pid=32355] ppid=32353 vsize=80812 CPUtime=408.47 cores=1,3,5,7
/proc/32355/stat : 32355 (march_hi) R 32353 32355 32355 0 -1 4202496 15008 0 0 0 40845 2 0 0 25 0 1 0 658269310 82751488 14742 33554432000 4194304 4955518 140734799665536 18446744073709551615 4224642 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/32355/statm: 20203 14742 104 186 0 20016 0
[pid=32245/tid=32252] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32252/stat : 32252 (python) S 32243 32245 30970 0 -1 4202560 26 933 0 0 0 0 0 0 18 0 9 0 658190855 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 7 0 0 0
[pid=32245/tid=32255] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32255/stat : 32255 (python) S 32243 32245 30970 0 -1 4202560 35 933 0 0 0 0 0 0 18 0 9 0 658190856 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
[pid=32245/tid=32258] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32258/stat : 32258 (python) S 32243 32245 30970 0 -1 4202560 24 933 0 0 0 0 0 0 22 0 9 0 658190857 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 7 0 0 0
[pid=32245/tid=32261] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32261/stat : 32261 (python) S 32243 32245 30970 0 -1 4202560 33 933 0 0 0 0 0 0 18 0 9 0 658190858 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
[pid=32245/tid=32311] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32311/stat : 32311 (python) S 32243 32245 30970 0 -1 4202560 34 933 0 0 0 0 0 0 18 0 9 0 658200681 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
[pid=32245/tid=32315] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32315/stat : 32315 (python) S 32243 32245 30970 0 -1 4202560 31 933 0 0 0 0 0 0 18 0 9 0 658200883 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
[pid=32245/tid=32336] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32336/stat : 32336 (python) S 32243 32245 30970 0 -1 4202560 33 933 0 0 0 0 0 0 18 0 9 0 658229997 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
[pid=32245/tid=32354] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32354/stat : 32354 (python) S 32243 32245 30970 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 9 0 658269310 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4780.75
Current children cumulated vsize (KiB) 5723948

[startup+1199.9 s]
/proc/loadavg: 9.32 9.26 9.11 10/406 32374
/proc/meminfo: memFree=10049240/32951124 swapFree=66050672/67111528
[pid=32245] ppid=32243 vsize=360988 CPUtime=0.61 cores=1,3,5,7
/proc/32245/stat : 32245 (python) S 32243 32245 30970 0 -1 4202496 16347 933 1 0 49 12 0 0 18 0 9 0 658190776 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/32245/statm: 90247 7245 1890 364 0 26144 0
[pid=32251] ppid=32245 vsize=288816 CPUtime=0.25 cores=1,3,5,7
/proc/32251/stat : 32251 (python) S 32245 32245 30970 0 -1 4202560 1165 0 0 0 13 12 0 0 19 0 2 0 658190855 295747584 5638 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/32251/statm: 72204 5638 416 364 0 8122 0
[pid=32253] ppid=32251 vsize=1017940 CPUtime=98.24 cores=1,3,5,7
/proc/32253/stat : 32253 (gnovelty+2) T 32251 32253 32253 0 -1 4202496 456 0 0 0 9824 0 0 0 25 0 1 0 658190855 1042370560 192 33554432000 134512640 135663704 4287541184 18446744073709551615 134545644 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/32253/statm: 254485 192 113 282 0 254201 0
[pid=32251/tid=32309] ppid=32245 vsize=288816 CPUtime=0 cores=1,3,5,7
/proc/32251/task/32309/stat : 32309 (python) S 32245 32245 30970 0 -1 4202560 42 0 0 0 0 0 0 0 19 0 2 0 658200680 295747584 5638 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604464183288832 0 0 -1 7 0 0 0
[pid=32254] ppid=32245 vsize=289416 CPUtime=2.44 cores=1,3,5,7
/proc/32254/stat : 32254 (python) S 32245 32245 30970 0 -1 4202560 1146 0 0 0 128 116 0 0 15 0 2 0 658190856 296361984 5765 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/32254/statm: 72354 5765 400 364 0 8269 0
[pid=32256] ppid=32254 vsize=173568 CPUtime=894.61 cores=1,3,5,7
/proc/32256/stat : 32256 (march_hi) R 32254 32256 32256 0 -1 4202496 31157 0 0 0 89456 5 0 0 25 0 1 0 658190856 177733632 30894 33554432000 4194304 4955518 140734524727856 18446744073709551615 4235344 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/32256/statm: 43392 30894 104 186 0 43205 0
[pid=32254/tid=32333] ppid=32245 vsize=289416 CPUtime=0 cores=1,3,5,7
/proc/32254/task/32333/stat : 32333 (python) S 32245 32245 30970 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 658229955 296361984 5765 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604470122893376 0 0 -1 3 0 0 0
[pid=32257] ppid=32245 vsize=299168 CPUtime=0.25 cores=1,3,5,7
/proc/32257/stat : 32257 (python) S 32245 32245 30970 0 -1 4202560 1077 0 0 0 16 9 0 0 19 0 2 0 658190857 306348032 5658 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/32257/statm: 74792 5658 400 364 0 10704 0
[pid=32259] ppid=32257 vsize=1017940 CPUtime=98.66 cores=1,3,5,7
/proc/32259/stat : 32259 (gnovelty+2) T 32257 32259 32259 0 -1 4202496 450 0 0 0 9866 0 0 0 25 0 1 0 658190857 1042370560 192 33554432000 134512640 135663704 4289339600 18446744073709551615 134524498 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/32259/statm: 254485 192 113 282 0 254201 0
[pid=32257/tid=32313] ppid=32245 vsize=299168 CPUtime=0 cores=1,3,5,7
/proc/32257/task/32313/stat : 32313 (python) S 32245 32245 30970 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 658200881 306348032 5658 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604457680737344 0 0 -1 7 0 0 0
[pid=32260] ppid=32245 vsize=310032 CPUtime=1.87 cores=1,3,5,7
/proc/32260/stat : 32260 (python) S 32245 32245 30970 0 -1 4202560 1159 0 0 0 100 87 0 0 15 0 2 0 658190858 317472768 5782 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/32260/statm: 77508 5782 400 364 0 13417 0
[pid=32262] ppid=32260 vsize=110412 CPUtime=684.55 cores=1,3,5,7
/proc/32262/stat : 32262 (march_hi) T 32260 32262 32262 0 -1 4202496 24276 0 0 0 68450 5 0 0 25 0 1 0 658190858 113061888 24015 33554432000 4194304 4955518 140736794816432 18446744073709551615 4224512 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/32262/statm: 27603 24015 104 186 0 27416 0
[pid=32260/tid=32334] ppid=32245 vsize=310032 CPUtime=0 cores=1,3,5,7
/proc/32260/task/32334/stat : 32334 (python) S 32245 32245 30970 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 658229996 317472768 5782 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604464367268992 0 0 -1 5 0 0 0
[pid=32310] ppid=32245 vsize=320288 CPUtime=2.72 cores=1,3,5,7
/proc/32310/stat : 32310 (python) S 32245 32245 30970 0 -1 4202560 1149 0 0 0 143 129 0 0 15 0 2 0 658200681 327974912 5786 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/32310/statm: 80072 5786 400 364 0 15978 0
[pid=32312] ppid=32310 vsize=173568 CPUtime=972.27 cores=1,3,5,7
/proc/32312/stat : 32312 (march_hi) T 32310 32312 32312 0 -1 4202496 33424 0 0 0 97220 7 0 0 25 0 1 0 658200681 177733632 33163 33554432000 4194304 4955518 140735196371296 18446744073709551615 4314282 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/32312/statm: 43392 33163 104 186 0 43205 0
[pid=32310/tid=32356] ppid=32245 vsize=320288 CPUtime=0 cores=1,3,5,7
/proc/32310/task/32356/stat : 32356 (python) S 32245 32245 30970 0 -1 4202560 45 0 0 0 0 0 0 0 15 0 2 0 658270022 327974912 5786 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604469906150336 0 0 -1 7 0 0 0
[pid=32314] ppid=32245 vsize=330016 CPUtime=2.2 cores=1,3,5,7
/proc/32314/stat : 32314 (python) S 32245 32245 30970 0 -1 4202560 1145 0 0 0 113 107 0 0 15 0 2 0 658200882 337936384 5690 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/32314/statm: 82504 5690 400 364 0 18407 0
[pid=32316] ppid=32314 vsize=129368 CPUtime=806 cores=1,3,5,7
/proc/32316/stat : 32316 (march_hi) R 32314 32316 32316 0 -1 4202496 28330 0 0 0 80595 5 0 0 25 0 1 0 658200883 132472832 28073 33554432000 4194304 4955518 140734991633520 18446744073709551615 4235311 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/32316/statm: 32342 28073 104 186 0 32155 0
[pid=32314/tid=32352] ppid=32245 vsize=330016 CPUtime=0 cores=1,3,5,7
/proc/32314/task/32352/stat : 32352 (python) S 32245 32245 30970 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 658269309 337936384 5690 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604469906151232 0 0 -1 1 0 0 0
[pid=32335] ppid=32245 vsize=340896 CPUtime=2.21 cores=1,3,5,7
/proc/32335/stat : 32335 (python) S 32245 32245 30970 0 -1 4202560 1164 0 0 0 117 104 0 0 15 0 2 0 658229997 349077504 5819 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/32335/statm: 85224 5819 400 364 0 21124 0
[pid=32337] ppid=32335 vsize=129368 CPUtime=805.44 cores=1,3,5,7
/proc/32337/stat : 32337 (march_hi) R 32335 32337 32337 0 -1 4202496 28281 0 0 0 80537 7 0 0 25 0 1 0 658229997 132472832 28019 33554432000 4194304 4955518 140736798769136 18446744073709551615 4290061 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/32337/statm: 32342 28019 104 186 0 32155 0
[pid=32335/tid=32367] ppid=32245 vsize=340896 CPUtime=0 cores=1,3,5,7
/proc/32335/task/32367/stat : 32367 (python) S 32245 32245 30970 0 -1 4202560 49 0 0 0 0 0 0 0 17 0 2 0 658288744 349077504 5819 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 514 18446604468630841280 0 0 -1 1 0 0 0
[pid=32353] ppid=32245 vsize=351352 CPUtime=1.16 cores=1,3,5,7
/proc/32353/stat : 32353 (python) S 32245 32245 30970 0 -1 4202560 1043 0 0 0 61 55 0 0 15 0 1 0 658269310 359784448 5848 33554432000 4194304 5683644 140737049898512 18446744073709551615 255377321542 0 0 16781312 514 9092679686730776839 0 0 17 5 0 0 0
/proc/32353/statm: 87838 5848 377 364 0 23735 0
[pid=32355] ppid=32353 vsize=80812 CPUtime=408.87 cores=1,3,5,7
/proc/32355/stat : 32355 (march_hi) R 32353 32355 32355 0 -1 4202496 15020 0 0 0 40885 2 0 0 25 0 1 0 658269310 82751488 14754 33554432000 4194304 4955518 140734799665536 18446744073709551615 4227671 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/32355/statm: 20203 14754 104 186 0 20016 0
[pid=32245/tid=32252] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32252/stat : 32252 (python) S 32243 32245 30970 0 -1 4202560 26 933 0 0 0 0 0 0 18 0 9 0 658190855 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 7 0 0 0
[pid=32245/tid=32255] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32255/stat : 32255 (python) S 32243 32245 30970 0 -1 4202560 35 933 0 0 0 0 0 0 18 0 9 0 658190856 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
[pid=32245/tid=32258] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32258/stat : 32258 (python) S 32243 32245 30970 0 -1 4202560 24 933 0 0 0 0 0 0 22 0 9 0 658190857 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 7 0 0 0
[pid=32245/tid=32261] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32261/stat : 32261 (python) S 32243 32245 30970 0 -1 4202560 33 933 0 0 0 0 0 0 18 0 9 0 658190858 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
[pid=32245/tid=32311] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32311/stat : 32311 (python) S 32243 32245 30970 0 -1 4202560 34 933 0 0 0 0 0 0 18 0 9 0 658200681 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
[pid=32245/tid=32315] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32315/stat : 32315 (python) S 32243 32245 30970 0 -1 4202560 31 933 0 0 0 0 0 0 18 0 9 0 658200883 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
[pid=32245/tid=32336] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32336/stat : 32336 (python) S 32243 32245 30970 0 -1 4202560 33 933 0 0 0 0 0 0 18 0 9 0 658229997 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
[pid=32245/tid=32354] ppid=32243 vsize=360988 CPUtime=0 cores=1,3,5,7
/proc/32245/task/32354/stat : 32354 (python) S 32243 32245 30970 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 9 0 658269310 369651712 7245 33554432000 4194304 5683644 140737049898512 18446744073709551615 255389125041 0 0 16781312 2 18446604466851893440 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4782.35
Current children cumulated vsize (KiB) 5723948

Child status: 0
Real time (s): 1199.98
CPU time (s): 4782.76
CPU user time (s): 4776.02
CPU system time (s): 6.74297
CPU usage (%): 398.569
Max. virtual memory (cumulated for all children) (KiB): 5723952

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4776.02
system time used= 6.74297
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 192165
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9651
involuntary context switches= 26707

runsolver used 4.74128 second user time and 9.80451 second system time

The end

Launcher Data 

Begin job on node135 at 2011-05-15 15:25:29
IDJOB=3378495
IDBENCH=82904
IDSOLVER=1859
FILE ID=node135/3378495-1305465929
RUNJOBID= node135-1305462647-30989
PBS_JOBID= 13324970
Free space on /tmp= 73312 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v650-c2769-S910125393-002.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378495-1305465929/watcher-3378495-1305465929 -o /tmp/evaluation-result-3378495-1305465929/solver-3378495-1305465929 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378495-1305465929.cnf -seed 736253521 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8567972e0c3ca849f27342fc5fce028c
RANDOM SEED=736253521

node135.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6640.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.78
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      10587012 kB
Buffers:        301136 kB
Cached:        6358160 kB
SwapCached:      50684 kB
Active:       11241248 kB
Inactive:     10857608 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      10587012 kB
SwapTotal:    67111528 kB
SwapFree:     66050672 kB
Dirty:           31884 kB
Writeback:           0 kB
AnonPages:    15394696 kB
Mapped:          28632 kB
Slab:           144788 kB
PageTables:      57712 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 83276736 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73280 MiB
End job on node135 at 2011-05-15 15:45:29