Trace number 3317208

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (TO) 4026.86 1300.1

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v15000-c63000-S594553190-015-UNKNOWN.cnf
MD5SUM0a98d6d9a7bdb797305aed69a94f9a69
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark355.73
Satisfiable
(Un)Satisfiability was proved
Number of variables15000
Number of clauses63000
Sum of the clauses size189000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 363000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.35/0.63	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.63	c INFO: solving HOME/instance-3317208-1304268200.cnf
0.35/0.63	c INFO: computing task features
0.54/0.86	c INFO: computed variable-clause graph statistics
0.54/0.87	c INFO: computed clause balance statistics
1.73/2.01	c INFO: computed variable balance statistics
2.63/2.97	c INFO: computed small-clause counts
2.63/2.98	c INFO: computed Horn-clause counts
3.04/3.32	c INFO: computed variable graph statistics
3.04/3.32	c INFO: collected features for HOME/instance-3317208-1304268200.cnf in 2.68 s
3.04/3.35	c INFO: running hybridGM3@0 for 97 with 4797 remaining (b = 0.46)
3.04/3.38	c INFO: running hybridGM3@0 for 97 with 4797 remaining (b = 0.28)
3.13/3.41	c INFO: running hybridGM3@0 for 97 with 4797 remaining (b = 0.20)
3.13/3.44	c INFO: running hybridGM3@0 for 194 with 4797 remaining (b = 0.28)
315.91/102.82	c INFO: running hybridGM3@0 for 97 with 4697 remaining (b = 0.13)
319.41/103.87	c INFO: running hybridGM3@0 for 97 with 4696 remaining (b = 0.13)
464.10/152.08	c INFO: running hybridGM3@0 for 97 with 4648 remaining (b = 0.13)
716.85/236.29	c INFO: running hybridGM3@98 for 97 with 4564 remaining (b = 0.13)
773.77/253.26	c INFO: running hybridGM3@0 for 194 with 4547 remaining (b = 0.22)
811.91/265.14	c INFO: running hybridGM3@98 for 97 with 4535 remaining (b = 0.10)
826.90/270.10	c INFO: running TNM@0 for 97 with 4530 remaining (b = 0.11)
1064.76/349.49	c INFO: running hybridGM3@100 for 97 with 4451 remaining (b = 0.09)
1267.42/408.39	c INFO: running hybridGM3@0 for 194 with 4392 remaining (b = 0.16)
1281.36/412.35	c INFO: running hybridGM3@0 for 194 with 4388 remaining (b = 0.12)
1486.38/476.73	c INFO: running TNM@96 for 291 with 4323 remaining (b = 0.16)
1513.98/485.91	c INFO: running gnovelty+2@0 for 97 with 4314 remaining (b = 0.08)
1868.06/597.25	c INFO: running hybridGM3@94 for 97 with 4203 remaining (b = 0.06)
2042.92/647.32	c INFO: running hybridGM3@102 for 97 with 4153 remaining (b = 0.06)
2126.46/675.31	c INFO: running hybridGM3@96 for 97 with 4125 remaining (b = 0.06)
2254.98/718.47	c INFO: running hybridGM3@0 for 194 with 4082 remaining (b = 0.10)
2390.62/760.53	c INFO: running hybridGM3@0 for 194 with 4040 remaining (b = 0.07)
2527.83/806.55	c INFO: running adaptg2wsat2009++@0 for 97 with 3994 remaining (b = 0.03)
2609.23/832.33	c INFO: running TNM@291 for 97 with 3968 remaining (b = 0.04)
2880.57/922.80	c INFO: running hybridGM3@299 for 97 with 3877 remaining (b = 0.03)
3002.28/963.58	c INFO: running TNM@394 for 194 with 3837 remaining (b = 0.05)
3016.86/968.47	c INFO: running hybridGM3@97 for 97 with 3832 remaining (b = 0.03)
3242.38/1044.04	c INFO: running hybridGM3@0 for 194 with 3756 remaining (b = 0.06)
3338.44/1071.61	c INFO: running hybridGM3@97 for 97 with 3729 remaining (b = 0.03)
3442.10/1104.75	c INFO: running hybridGM3@392 for 97 with 3695 remaining (b = 0.02)
3646.06/1172.97	c INFO: running TNM@591 for 776 with 3627 remaining (b = 0.18)
3789.29/1220.89	c INFO: running hybridGM3@0 for 194 with 3579 remaining (b = 0.04)
3885.63/1252.93	c INFO: running hybridGM3@0 for 194 with 3547 remaining (b = 0.03)
4003.16/1292.11	c INFO: running TNM@0 for 97 with 3508 remaining (b = 0.02)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317208-1304268200/watcher-3317208-1304268200 -o /tmp/evaluation-result-3317208-1304268200/solver-3317208-1304268200 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317208-1304268200.cnf -seed 1544105914 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.22 7.75 5.02 6/250 17339
/proc/meminfo: memFree=30206428/32951124 swapFree=67111364/67111528
[pid=17339] ppid=17337 vsize=15380 CPUtime=0 cores=0,2,4,6
/proc/17339/stat : 17339 (python) S 17337 17339 16913 0 -1 4202496 811 0 0 0 0 0 0 0 17 0 1 0 538418027 15749120 696 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244346880 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/17339/statm: 3845 696 334 364 0 332 0

[startup+0.0492481 s]
/proc/loadavg: 8.22 7.75 5.02 6/250 17339
/proc/meminfo: memFree=30206428/32951124 swapFree=67111364/67111528
[pid=17339] ppid=17337 vsize=21100 CPUtime=0.02 cores=0,2,4,6
/proc/17339/stat : 17339 (python) R 17337 17339 16913 0 -1 4202496 1315 0 0 0 2 0 0 0 17 0 1 0 538418027 21606400 1165 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244346880 0 2147483391 16781312 2 0 0 0 17 4 0 0 0
/proc/17339/statm: 5275 1165 428 364 0 722 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 21100

[startup+0.100242 s]
/proc/loadavg: 8.22 7.75 5.02 6/250 17339
/proc/meminfo: memFree=30206428/32951124 swapFree=67111364/67111528
[pid=17339] ppid=17337 vsize=93960 CPUtime=0.05 cores=0,2,4,6
/proc/17339/stat : 17339 (python) S 17337 17339 16913 0 -1 4202496 2694 0 0 0 4 1 0 0 17 0 1 0 538418027 96215040 2407 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244346880 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/17339/statm: 23490 2407 764 364 0 1622 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 93960

[startup+0.300171 s]
/proc/loadavg: 8.22 7.75 5.02 6/250 17339
/proc/meminfo: memFree=30206428/32951124 swapFree=67111364/67111528
[pid=17339] ppid=17337 vsize=135748 CPUtime=0.17 cores=0,2,4,6
/proc/17339/stat : 17339 (python) R 17337 17339 16913 0 -1 4202496 5964 937 1 0 13 4 0 0 18 0 1 0 538418027 139005952 4151 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244344837 0 2147483391 16781312 2 18446744071699199166 0 0 17 0 0 0 0
/proc/17339/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135748

[startup+0.701099 s]
/proc/loadavg: 8.22 7.75 5.02 6/250 17339
/proc/meminfo: memFree=30206428/32951124 swapFree=67111364/67111528
[pid=17339] ppid=17337 vsize=282984 CPUtime=0.44 cores=0,2,4,6
/proc/17339/stat : 17339 (python) R 17337 17339 16913 0 -1 4202496 10485 937 1 0 34 10 0 0 16 0 1 0 538418027 289775616 8058 33554432000 4194304 5683644 140734634892432 18446744073709551615 4602370 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/17339/statm: 70746 8058 1761 364 0 6670 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 282984

[startup+1.50096 s]
/proc/loadavg: 8.22 7.75 5.02 7/251 17344
/proc/meminfo: memFree=30170460/32951124 swapFree=67111364/67111528
[pid=17339] ppid=17337 vsize=292744 CPUtime=1.23 cores=0,2,4,6
/proc/17339/stat : 17339 (python) R 17337 17339 16913 0 -1 4202496 14666 937 1 0 113 10 0 0 17 0 1 0 538418027 299769856 10571 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244003885 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/17339/statm: 73186 10571 1826 364 0 9110 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 292744

[startup+3.10068 s]
/proc/loadavg: 8.22 7.75 5.02 7/251 17344
/proc/meminfo: memFree=30170088/32951124 swapFree=67111364/67111528
[pid=17339] ppid=17337 vsize=292744 CPUtime=2.83 cores=0,2,4,6
/proc/17339/stat : 17339 (python) R 17337 17339 16913 0 -1 4202496 14673 937 1 0 273 10 0 0 19 0 1 0 538418027 299769856 10578 33554432000 4194304 5683644 140734634892432 18446744073709551615 47128682252428 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/17339/statm: 73186 10578 1833 364 0 9110 0
Current children cumulated CPU time (s) 2.83
Current children cumulated vsize (KiB) 292744

[startup+6.30108 s]
/proc/loadavg: 8.12 7.74 5.03 10/263 17356
/proc/meminfo: memFree=30067444/32951124 swapFree=67111364/67111528
[pid=17339] ppid=17337 vsize=326328 CPUtime=3.17 cores=0,2,4,6
/proc/17339/stat : 17339 (python) S 17337 17339 16913 0 -1 4202496 17541 937 1 0 306 11 0 0 21 0 5 0 538418027 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/17339/statm: 81582 8779 1881 364 0 17491 0
[pid=17345] ppid=17339 vsize=285800 CPUtime=0 cores=0,2,4,6
/proc/17345/stat : 17345 (python) S 17339 17339 16913 0 -1 4202560 735 0 0 0 0 0 0 0 15 0 1 0 538418362 292659200 7103 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368911 0 0 16781312 514 2642893630078943492 0 0 17 0 0 0 0
/proc/17345/statm: 71450 7103 355 364 0 7368 0
[pid=17347] ppid=17345 vsize=268424 CPUtime=1.5 cores=0,2,4,6
/proc/17347/stat : 17347 (hybridGM3) R 17345 17347 17347 0 -1 4202496 6189 0 0 0 149 1 0 0 25 0 1 0 538418363 274866176 5923 33554432000 134512640 135204385 4289495440 18446744073709551615 134537608 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/17347/statm: 67106 5923 64 169 0 66936 0
[pid=17348] ppid=17339 vsize=295788 CPUtime=0 cores=0,2,4,6
/proc/17348/stat : 17348 (python) S 17339 17339 16913 0 -1 4202560 744 0 0 0 0 0 0 0 18 0 1 0 538418365 302886912 7236 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 3810370744492196100 0 0 17 6 0 0 0
/proc/17348/statm: 73947 7236 349 364 0 9862 0
[pid=17350] ppid=17348 vsize=268428 CPUtime=2.88 cores=0,2,4,6
/proc/17350/stat : 17350 (hybridGM3) R 17348 17350 17350 0 -1 4202496 6186 0 0 0 288 0 0 0 25 0 1 0 538418366 274870272 5920 33554432000 134512640 135204385 4289632880 18446744073709551615 134537249 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/17350/statm: 67107 5920 64 169 0 66937 0
[pid=17351] ppid=17339 vsize=306044 CPUtime=0 cores=0,2,4,6
/proc/17351/stat : 17351 (python) S 17339 17339 16913 0 -1 4202560 748 0 0 0 0 0 0 0 15 0 1 0 538418368 313389056 7240 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 3961101144276566273 0 0 17 0 0 0 0
/proc/17351/statm: 76511 7240 349 364 0 12423 0
[pid=17355] ppid=17351 vsize=268424 CPUtime=2.84 cores=0,2,4,6
/proc/17355/stat : 17355 (hybridGM3) R 17351 17355 17355 0 -1 4202496 6184 0 0 0 284 0 0 0 25 0 1 0 538418372 274866176 5920 33554432000 134512640 135204385 4293634992 18446744073709551615 134549096 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17355/statm: 67106 5920 64 169 0 66936 0
[pid=17353] ppid=17339 vsize=316368 CPUtime=0 cores=0,2,4,6
/proc/17353/stat : 17353 (python) S 17339 17339 16913 0 -1 4202560 762 0 0 0 0 0 0 0 15 0 1 0 538418371 323960832 7244 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 2642897478369640708 0 0 17 0 0 0 0
/proc/17353/statm: 79092 7244 349 364 0 15001 0
[pid=17356] ppid=17353 vsize=268424 CPUtime=1.42 cores=0,2,4,6
/proc/17356/stat : 17356 (hybridGM3) R 17353 17356 17356 0 -1 4202496 6184 0 0 0 142 0 0 0 25 0 1 0 538418372 274866176 5921 33554432000 134512640 135204385 4290722160 18446744073709551615 134547438 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/17356/statm: 67106 5921 64 169 0 66936 0
[pid=17339/tid=17346] ppid=17337 vsize=326328 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17346/stat : 17346 (python) S 17337 17339 16913 0 -1 4202560 26 937 0 0 0 0 0 0 22 0 5 0 538418363 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17349] ppid=17337 vsize=326328 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17349/stat : 17349 (python) S 17337 17339 16913 0 -1 4202560 23 937 0 0 0 0 0 0 24 0 5 0 538418366 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 6 0 0 0
[pid=17339/tid=17352] ppid=17337 vsize=326328 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17352/stat : 17352 (python) S 17337 17339 16913 0 -1 4202560 31 937 0 0 0 0 0 0 20 0 5 0 538418369 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17354] ppid=17337 vsize=326328 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17354/stat : 17354 (python) S 17337 17339 16913 0 -1 4202560 20 937 0 0 0 0 0 0 22 0 5 0 538418372 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.81
Current children cumulated vsize (KiB) 2604028

[startup+12.701 s]
/proc/loadavg: 8.19 7.76 5.05 11/263 17356
/proc/meminfo: memFree=30066196/32951124 swapFree=67111364/67111528
[pid=17339] ppid=17337 vsize=326328 CPUtime=3.17 cores=0,2,4,6
/proc/17339/stat : 17339 (python) S 17337 17339 16913 0 -1 4202496 17541 937 1 0 306 11 0 0 21 0 5 0 538418027 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/17339/statm: 81582 8779 1881 364 0 17491 0
[pid=17345] ppid=17339 vsize=285800 CPUtime=0 cores=0,2,4,6
/proc/17345/stat : 17345 (python) S 17339 17339 16913 0 -1 4202560 897 0 0 0 0 0 0 0 15 0 1 0 538418362 292659200 7126 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368911 0 0 16781312 514 2642893630078943492 0 0 17 0 0 0 0
/proc/17345/statm: 71450 7126 378 364 0 7368 0
[pid=17347] ppid=17345 vsize=268424 CPUtime=4.7 cores=0,2,4,6
/proc/17347/stat : 17347 (hybridGM3) R 17345 17347 17347 0 -1 4202496 6189 0 0 0 469 1 0 0 25 0 1 0 538418363 274866176 5923 33554432000 134512640 135204385 4289495440 18446744073709551615 134537272 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/17347/statm: 67106 5923 64 169 0 66936 0
[pid=17348] ppid=17339 vsize=295788 CPUtime=0 cores=0,2,4,6
/proc/17348/stat : 17348 (python) S 17339 17339 16913 0 -1 4202560 907 0 0 0 0 0 0 0 18 0 1 0 538418365 302886912 7258 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 3810370744492196100 0 0 17 6 0 0 0
/proc/17348/statm: 73947 7258 371 364 0 9862 0
[pid=17350] ppid=17348 vsize=268428 CPUtime=9.23 cores=0,2,4,6
/proc/17350/stat : 17350 (hybridGM3) R 17348 17350 17350 0 -1 4202496 6187 0 0 0 923 0 0 0 25 0 1 0 538418366 274870272 5921 33554432000 134512640 135204385 4289632880 18446744073709551615 134536646 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/17350/statm: 67107 5921 64 169 0 66937 0
[pid=17351] ppid=17339 vsize=306044 CPUtime=0 cores=0,2,4,6
/proc/17351/stat : 17351 (python) S 17339 17339 16913 0 -1 4202560 903 0 0 0 0 0 0 0 15 0 1 0 538418368 313389056 7263 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 3961101144276566273 0 0 17 0 0 0 0
/proc/17351/statm: 76511 7263 372 364 0 12423 0
[pid=17355] ppid=17351 vsize=268424 CPUtime=9.24 cores=0,2,4,6
/proc/17355/stat : 17355 (hybridGM3) R 17351 17355 17355 0 -1 4202496 6185 0 0 0 924 0 0 0 25 0 1 0 538418372 274866176 5921 33554432000 134512640 135204385 4293634992 18446744073709551615 134547471 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17355/statm: 67106 5921 64 169 0 66936 0
[pid=17353] ppid=17339 vsize=316368 CPUtime=0 cores=0,2,4,6
/proc/17353/stat : 17353 (python) S 17339 17339 16913 0 -1 4202560 936 0 0 0 0 0 0 0 15 0 1 0 538418371 323960832 7267 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 2642897478369640708 0 0 17 0 0 0 0
/proc/17353/statm: 79092 7267 372 364 0 15001 0
[pid=17356] ppid=17353 vsize=268424 CPUtime=4.62 cores=0,2,4,6
/proc/17356/stat : 17356 (hybridGM3) R 17353 17356 17356 0 -1 4202496 6184 0 0 0 462 0 0 0 25 0 1 0 538418372 274866176 5921 33554432000 134512640 135204385 4290722160 18446744073709551615 134537292 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/17356/statm: 67106 5921 64 169 0 66936 0
[pid=17339/tid=17346] ppid=17337 vsize=326328 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17346/stat : 17346 (python) S 17337 17339 16913 0 -1 4202560 26 937 0 0 0 0 0 0 22 0 5 0 538418363 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17349] ppid=17337 vsize=326328 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17349/stat : 17349 (python) S 17337 17339 16913 0 -1 4202560 23 937 0 0 0 0 0 0 24 0 5 0 538418366 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 6 0 0 0
[pid=17339/tid=17352] ppid=17337 vsize=326328 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17352/stat : 17352 (python) S 17337 17339 16913 0 -1 4202560 31 937 0 0 0 0 0 0 20 0 5 0 538418369 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17354] ppid=17337 vsize=326328 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17354/stat : 17354 (python) S 17337 17339 16913 0 -1 4202560 20 937 0 0 0 0 0 0 22 0 5 0 538418372 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 30.96
Current children cumulated vsize (KiB) 2604028

[startup+25.5007 s]
/proc/loadavg: 8.37 7.82 5.11 10/263 17356
/proc/meminfo: memFree=30066636/32951124 swapFree=67111364/67111528
[pid=17339] ppid=17337 vsize=326328 CPUtime=3.17 cores=0,2,4,6
/proc/17339/stat : 17339 (python) S 17337 17339 16913 0 -1 4202496 17541 937 1 0 306 11 0 0 21 0 5 0 538418027 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/17339/statm: 81582 8779 1881 364 0 17491 0
[pid=17345] ppid=17339 vsize=285800 CPUtime=0.02 cores=0,2,4,6
/proc/17345/stat : 17345 (python) S 17339 17339 16913 0 -1 4202560 902 0 0 0 1 1 0 0 15 0 1 0 538418362 292659200 7126 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368911 0 0 16781312 514 18254162413081297156 0 0 17 0 0 0 0
/proc/17345/statm: 71450 7126 378 364 0 7368 0
[pid=17347] ppid=17345 vsize=268424 CPUtime=11.1 cores=0,2,4,6
/proc/17347/stat : 17347 (hybridGM3) R 17345 17347 17347 0 -1 4202496 6191 0 0 0 1109 1 0 0 25 0 1 0 538418363 274866176 5925 33554432000 134512640 135204385 4289495440 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/17347/statm: 67106 5925 64 169 0 66936 0
[pid=17348] ppid=17339 vsize=295788 CPUtime=0.02 cores=0,2,4,6
/proc/17348/stat : 17348 (python) S 17339 17339 16913 0 -1 4202560 914 0 0 0 1 1 0 0 18 0 1 0 538418365 302886912 7259 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 18254162413081297156 0 0 17 6 0 0 0
/proc/17348/statm: 73947 7259 372 364 0 9862 0
[pid=17350] ppid=17348 vsize=268428 CPUtime=21.91 cores=0,2,4,6
/proc/17350/stat : 17350 (hybridGM3) R 17348 17350 17350 0 -1 4202496 6187 0 0 0 2191 0 0 0 25 0 1 0 538418366 274870272 5921 33554432000 134512640 135204385 4289632880 18446744073709551615 134547366 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/17350/statm: 67107 5921 64 169 0 66937 0
[pid=17351] ppid=17339 vsize=306044 CPUtime=0.02 cores=0,2,4,6
/proc/17351/stat : 17351 (python) S 17339 17339 16913 0 -1 4202560 907 0 0 0 1 1 0 0 15 0 1 0 538418368 313389056 7263 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 18254162413081297153 0 0 17 0 0 0 0
/proc/17351/statm: 76511 7263 372 364 0 12423 0
[pid=17355] ppid=17351 vsize=268424 CPUtime=22.04 cores=0,2,4,6
/proc/17355/stat : 17355 (hybridGM3) R 17351 17355 17355 0 -1 4202496 6188 0 0 0 2204 0 0 0 25 0 1 0 538418372 274866176 5924 33554432000 134512640 135204385 4293634992 18446744073709551615 134537249 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17355/statm: 67106 5924 64 169 0 66936 0
[pid=17353] ppid=17339 vsize=316368 CPUtime=0.02 cores=0,2,4,6
/proc/17353/stat : 17353 (python) S 17339 17339 16913 0 -1 4202560 941 0 0 0 1 1 0 0 15 0 1 0 538418371 323960832 7267 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 18254162413081297156 0 0 17 0 0 0 0
/proc/17353/statm: 79092 7267 372 364 0 15001 0
[pid=17356] ppid=17353 vsize=268424 CPUtime=11.02 cores=0,2,4,6
/proc/17356/stat : 17356 (hybridGM3) R 17353 17356 17356 0 -1 4202496 6184 0 0 0 1102 0 0 0 25 0 1 0 538418372 274866176 5921 33554432000 134512640 135204385 4290722160 18446744073709551615 134546144 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/17356/statm: 67106 5921 64 169 0 66936 0
[pid=17339/tid=17346] ppid=17337 vsize=326328 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17346/stat : 17346 (python) S 17337 17339 16913 0 -1 4202560 26 937 0 0 0 0 0 0 22 0 5 0 538418363 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17349] ppid=17337 vsize=326328 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17349/stat : 17349 (python) S 17337 17339 16913 0 -1 4202560 23 937 0 0 0 0 0 0 24 0 5 0 538418366 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 6 0 0 0
[pid=17339/tid=17352] ppid=17337 vsize=326328 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17352/stat : 17352 (python) S 17337 17339 16913 0 -1 4202560 31 937 0 0 0 0 0 0 20 0 5 0 538418369 334159872 8779 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0

################
# More data... #
################

/proc/17347/statm: 67106 5927 64 169 0 66936 0
[pid=17345/tid=17376] ppid=17339 vsize=296044 CPUtime=0 cores=0,2,4,6
/proc/17345/task/17376/stat : 17376 (python) S 17339 17339 16913 0 -1 4202560 50 0 0 0 0 0 0 0 15 0 2 0 538433233 303149056 7164 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604444938105280 0 0 -1 0 0 0 0
[pid=17348] ppid=17339 vsize=295788 CPUtime=0.67 cores=0,2,4,6
/proc/17348/stat : 17348 (python) S 17339 17339 16913 0 -1 4202560 1153 0 0 0 35 32 0 0 18 0 2 0 538418365 302886912 7288 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/17348/statm: 73947 7288 400 364 0 9862 0
[pid=17350] ppid=17348 vsize=268428 CPUtime=293.34 cores=0,2,4,6
/proc/17350/stat : 17350 (hybridGM3) T 17348 17350 17350 0 -1 4202496 6189 0 0 0 29331 3 0 0 25 0 1 0 538418366 274870272 5923 33554432000 134512640 135204385 4289632880 18446744073709551615 134536676 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/17350/statm: 67107 5923 64 169 0 66937 0
[pid=17348/tid=17371] ppid=17339 vsize=295788 CPUtime=0 cores=0,2,4,6
/proc/17348/task/17371/stat : 17371 (python) S 17339 17339 16913 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 538428306 302886912 7288 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604449504897024 0 0 -1 6 0 0 0
[pid=17351] ppid=17339 vsize=306044 CPUtime=0.67 cores=0,2,4,6
/proc/17351/stat : 17351 (python) S 17339 17339 16913 0 -1 4202560 1156 0 0 0 37 30 0 0 15 0 2 0 538418368 313389056 7292 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17351/statm: 76511 7292 400 364 0 12423 0
[pid=17355] ppid=17351 vsize=268424 CPUtime=295.02 cores=0,2,4,6
/proc/17355/stat : 17355 (hybridGM3) T 17351 17355 17355 0 -1 4202496 6188 0 0 0 29501 1 0 0 25 0 1 0 538418372 274866176 5924 33554432000 134512640 135204385 4293634992 18446744073709551615 134537502 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/17355/statm: 67106 5924 64 169 0 66936 0
[pid=17351/tid=17372] ppid=17339 vsize=306044 CPUtime=0 cores=0,2,4,6
/proc/17351/task/17372/stat : 17372 (python) S 17339 17339 16913 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 538428412 313389056 7292 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604453834958976 0 0 -1 0 0 0 0
[pid=17353] ppid=17339 vsize=316368 CPUtime=0.58 cores=0,2,4,6
/proc/17353/stat : 17353 (python) S 17339 17339 16913 0 -1 4202560 1169 0 0 0 33 25 0 0 15 0 2 0 538418371 323960832 7296 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17353/statm: 79092 7296 400 364 0 15001 0
[pid=17356] ppid=17353 vsize=268424 CPUtime=294.26 cores=0,2,4,6
/proc/17356/stat : 17356 (hybridGM3) T 17353 17356 17356 0 -1 4202496 6188 0 0 0 29425 1 0 0 25 0 1 0 538418372 274866176 5925 33554432000 134512640 135204385 4290722160 18446744073709551615 134547366 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/17356/statm: 67106 5925 64 169 0 66936 0
[pid=17353/tid=17383] ppid=17339 vsize=316368 CPUtime=0 cores=0,2,4,6
/proc/17353/task/17383/stat : 17383 (python) S 17339 17339 16913 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 538441654 323960832 7296 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604444938106176 0 0 -1 4 0 0 0
[pid=17396] ppid=17339 vsize=326612 CPUtime=0.6 cores=0,2,4,6
/proc/17396/stat : 17396 (python) S 17339 17339 16913 0 -1 4202560 1189 0 0 0 32 28 0 0 15 0 2 0 538443354 334450688 7367 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17396/statm: 81653 7367 399 364 0 17559 0
[pid=17398] ppid=17396 vsize=268424 CPUtime=295.69 cores=0,2,4,6
/proc/17398/stat : 17398 (hybridGM3) T 17396 17398 17398 0 -1 4202496 6192 0 0 0 29565 4 0 0 25 0 1 0 538443354 274866176 5926 33554432000 134512640 135204385 4289994528 18446744073709551615 134545866 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/17398/statm: 67106 5926 64 169 0 66936 0
[pid=17396/tid=17525] ppid=17339 vsize=326612 CPUtime=0 cores=0,2,4,6
/proc/17396/task/17525/stat : 17525 (python) S 17339 17339 16913 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 538466616 334450688 7367 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604453888088256 0 0 -1 0 0 0 0
[pid=17443] ppid=17339 vsize=337424 CPUtime=1.72 cores=0,2,4,6
/proc/17443/stat : 17443 (python) S 17339 17339 16913 0 -1 4202560 1172 0 0 0 98 74 0 0 18 0 2 0 538445037 345522176 7372 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17443/statm: 84356 7372 400 364 0 20259 0
[pid=17445] ppid=17443 vsize=457020 CPUtime=750.02 cores=0,2,4,6
/proc/17445/stat : 17445 (TNM) R 17443 17445 17445 0 -1 4202496 2778 0 0 0 74997 5 0 0 25 0 1 0 538445038 467988480 2516 33554432000 134512640 135008748 4288626032 18446744073709551615 134529176 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/17445/statm: 114255 2516 46 122 0 113155 0
[pid=17443/tid=17469] ppid=17339 vsize=337424 CPUtime=0 cores=0,2,4,6
/proc/17443/task/17469/stat : 17469 (python) S 17339 17339 16913 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 538459259 345522176 7372 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604453892440128 0 0 -1 2 0 0 0
[pid=17466] ppid=17339 vsize=347336 CPUtime=1.18 cores=0,2,4,6
/proc/17466/stat : 17466 (python) S 17339 17339 16913 0 -1 4202560 1233 0 0 0 67 51 0 0 15 0 2 0 538458866 355672064 7400 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17466/statm: 86834 7400 406 364 0 22734 0
[pid=17468] ppid=17466 vsize=268420 CPUtime=489.54 cores=0,2,4,6
/proc/17468/stat : 17468 (hybridGM3) T 17466 17468 17468 0 -1 4202496 6194 0 0 0 48952 2 0 0 25 0 1 0 538458867 274862080 5926 33554432000 134512640 135204385 4294836368 18446744073709551615 134537272 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/17468/statm: 67105 5926 64 169 0 66935 0
[pid=17466/tid=17542] ppid=17339 vsize=347336 CPUtime=0 cores=0,2,4,6
/proc/17466/task/17542/stat : 17542 (python) S 17339 17339 16913 0 -1 4202560 45 0 0 0 0 0 0 0 15 0 2 0 538482756 355672064 7400 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604453887481088 0 0 -1 0 0 0 0
[pid=17470] ppid=17339 vsize=357380 CPUtime=0.66 cores=0,2,4,6
/proc/17470/stat : 17470 (python) S 17339 17339 16913 0 -1 4202560 1213 0 0 0 38 28 0 0 15 0 2 0 538459263 365957120 7402 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17470/statm: 89345 7402 405 364 0 25242 0
[pid=17472] ppid=17470 vsize=268420 CPUtime=294.5 cores=0,2,4,6
/proc/17472/stat : 17472 (hybridGM3) T 17470 17472 17472 0 -1 4202496 6195 0 0 0 29449 1 0 0 25 0 1 0 538459263 274862080 5925 33554432000 134512640 135204385 4294692640 18446744073709551615 134537224 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/17472/statm: 67105 5925 64 169 0 66935 0
[pid=17470/tid=17545] ppid=17339 vsize=357380 CPUtime=0 cores=0,2,4,6
/proc/17470/task/17545/stat : 17545 (python) S 17339 17339 16913 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 538485555 365957120 7402 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604453869397824 0 0 -1 0 0 0 0
[pid=17526] ppid=17339 vsize=367832 CPUtime=0.19 cores=0,2,4,6
/proc/17526/stat : 17526 (python) S 17339 17339 16913 0 -1 4202560 1122 0 0 0 10 9 0 0 19 0 2 0 538466619 376659968 7431 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17526/statm: 91958 7431 399 364 0 27852 0
[pid=17528] ppid=17526 vsize=1021956 CPUtime=97.22 cores=0,2,4,6
/proc/17528/stat : 17528 (gnovelty+2) T 17526 17528 17528 0 -1 4202496 1715 0 0 0 9722 0 0 0 25 0 1 0 538466619 1046482944 1449 33554432000 134512640 135663704 4289171200 18446744073709551615 134546752 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/17528/statm: 255489 1449 113 282 0 255205 0
[pid=17526/tid=17540] ppid=17339 vsize=367832 CPUtime=0 cores=0,2,4,6
/proc/17526/task/17540/stat : 17540 (python) S 17339 17339 16913 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538477749 376659968 7431 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604451278344320 0 0 -1 0 0 0 0
[pid=17547] ppid=17339 vsize=378232 CPUtime=0.7 cores=0,2,4,6
/proc/17547/stat : 17547 (python) S 17339 17339 16913 0 -1 4202560 1190 0 0 0 37 33 0 0 15 0 2 0 538489874 387309568 7457 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/17547/statm: 94558 7457 409 364 0 30449 0
[pid=17549] ppid=17547 vsize=268424 CPUtime=294.67 cores=0,2,4,6
/proc/17549/stat : 17549 (hybridGM3) T 17547 17549 17549 0 -1 4202496 6190 0 0 0 29465 2 0 0 25 0 1 0 538489875 274866176 5925 33554432000 134512640 135204385 4290626400 18446744073709551615 134546059 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/17549/statm: 67106 5925 64 169 0 66936 0
[pid=17547/tid=17658] ppid=17339 vsize=378232 CPUtime=0 cores=0,2,4,6
/proc/17547/task/17658/stat : 17658 (python) S 17339 17339 16913 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 538514871 387309568 7457 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604453639625664 0 0 -1 6 0 0 0
[pid=17551] ppid=17339 vsize=388496 CPUtime=0.83 cores=0,2,4,6
/proc/17551/stat : 17551 (python) S 17339 17339 16913 0 -1 4202560 1206 0 0 0 46 37 0 0 15 0 2 0 538494080 397819904 7461 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17551/statm: 97124 7461 409 364 0 33012 0
[pid=17553] ppid=17551 vsize=268428 CPUtime=293.97 cores=0,2,4,6
/proc/17553/stat : 17553 (hybridGM3) T 17551 17553 17553 0 -1 4202496 6194 0 0 0 29394 3 0 0 25 0 1 0 538494081 274870272 5924 33554432000 134512640 135204385 4291344560 18446744073709551615 134536663 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/17553/statm: 67107 5924 64 169 0 66937 0
[pid=17551/tid=17674] ppid=17339 vsize=388496 CPUtime=0 cores=0,2,4,6
/proc/17551/task/17674/stat : 17674 (python) S 17339 17339 16913 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 538525185 397819904 7461 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604453977366784 0 0 -1 0 0 0 0
[pid=17555] ppid=17339 vsize=398880 CPUtime=0.2 cores=0,2,4,6
/proc/17555/stat : 17555 (python) S 17339 17339 16913 0 -1 4202560 1152 0 0 0 10 10 0 0 19 0 2 0 538498683 408453120 7456 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/17555/statm: 99720 7456 400 364 0 35605 0
[pid=17557] ppid=17555 vsize=422740 CPUtime=102.84 cores=0,2,4,6
/proc/17557/stat : 17557 (adaptg2wsat2009) T 17555 17557 17557 0 -1 4202496 2899 0 0 0 10283 1 0 0 25 0 1 0 538498683 432885760 2628 33554432000 134512640 135107411 4293759760 18446744073709551615 134531817 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/17557/statm: 105685 2628 59 146 0 104561 0
[pid=17555/tid=17648] ppid=17339 vsize=398880 CPUtime=0 cores=0,2,4,6
/proc/17555/task/17648/stat : 17648 (python) S 17339 17339 16913 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 538510304 408453120 7456 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604453846169728 0 0 -1 6 0 0 0
[pid=17661] ppid=17339 vsize=409012 CPUtime=0.38 cores=0,2,4,6
/proc/17661/stat : 17661 (python) S 17339 17339 16913 0 -1 4202560 1189 0 0 0 21 17 0 0 17 0 2 0 538522431 418828288 7487 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17661/statm: 102253 7487 399 364 0 38135 0
[pid=17663] ppid=17661 vsize=268424 CPUtime=195.39 cores=0,2,4,6
/proc/17663/stat : 17663 (hybridGM3) T 17661 17663 17663 0 -1 4202496 6189 0 0 0 19539 0 0 0 25 0 1 0 538522432 274866176 5923 33554432000 134512640 135204385 4286916176 18446744073709551615 134546082 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/17663/statm: 67106 5923 64 169 0 66936 0
[pid=17661/tid=17684] ppid=17339 vsize=409012 CPUtime=0 cores=0,2,4,6
/proc/17661/task/17684/stat : 17684 (python) S 17339 17339 16913 0 -1 4202560 49 0 0 0 0 0 0 0 17 0 2 0 538543317 418828288 7487 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 514 18446604453977369472 0 0 -1 0 0 0 0
[pid=17680] ppid=17339 vsize=419428 CPUtime=0.13 cores=0,2,4,6
/proc/17680/stat : 17680 (python) S 17339 17339 16913 0 -1 4202560 1008 0 0 0 7 6 0 0 15 0 1 0 538540116 429494272 7463 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 18254162413081297154 0 0 17 0 0 0 0
/proc/17680/statm: 104857 7463 372 364 0 40736 0
[pid=17682] ppid=17680 vsize=268428 CPUtime=78.58 cores=0,2,4,6
/proc/17682/stat : 17682 (hybridGM3) R 17680 17682 17682 0 -1 4202496 6190 0 0 0 7856 2 0 0 25 0 1 0 538540117 274870272 5922 33554432000 134512640 135204385 4292684032 18446744073709551615 134545859 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/17682/statm: 67107 5922 64 169 0 66937 0
[pid=17685] ppid=17339 vsize=429836 CPUtime=0.08 cores=0,2,4,6
/proc/17685/stat : 17685 (python) S 17339 17339 16913 0 -1 4202560 1067 0 0 0 4 4 0 0 15 0 1 0 538543320 440152064 7474 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 18254162413081297155 0 0 17 0 0 0 0
/proc/17685/statm: 107459 7474 379 364 0 43335 0
[pid=17687] ppid=17685 vsize=268420 CPUtime=35.5 cores=0,2,4,6
/proc/17687/stat : 17687 (hybridGM3) R 17685 17687 17687 0 -1 4202496 6199 0 0 0 3550 0 0 0 25 0 1 0 538543321 274862080 5921 33554432000 134512640 135204385 4289344464 18446744073709551615 134537565 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17687/statm: 67105 5921 64 169 0 66935 0
[pid=17692] ppid=17339 vsize=439780 CPUtime=0 cores=0,2,4,6
/proc/17692/stat : 17692 (python) S 17339 17339 16913 0 -1 4202560 1036 0 0 0 0 0 0 0 18 0 1 0 538547238 450334720 7476 33554432000 4194304 5683644 140734634892432 18446744073709551615 228244368966 0 0 16781312 514 5042241307141964035 0 0 17 0 0 0 0
/proc/17692/statm: 109945 7476 377 364 0 45818 0
[pid=17694] ppid=17692 vsize=457020 CPUtime=3.95 cores=0,2,4,6
/proc/17694/stat : 17694 (TNM) R 17692 17694 17694 0 -1 4202496 2792 0 0 0 395 0 0 0 25 0 1 0 538547239 467988480 2515 33554432000 134512640 135008748 4294392864 18446744073709551615 134527553 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17694/statm: 114255 2515 46 122 0 113155 0
[pid=17339/tid=17346] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17346/stat : 17346 (python) S 17337 17339 16913 0 -1 4202560 30 937 0 0 0 0 0 0 18 0 17 0 538418363 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17349] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17349/stat : 17349 (python) S 17337 17339 16913 0 -1 4202560 29 937 0 0 0 0 0 0 18 0 17 0 538418366 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 6 0 0 0
[pid=17339/tid=17352] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17352/stat : 17352 (python) S 17337 17339 16913 0 -1 4202560 35 937 0 0 0 0 0 0 18 0 17 0 538418369 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17354] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17354/stat : 17354 (python) S 17337 17339 16913 0 -1 4202560 20 937 0 0 0 0 0 0 18 0 17 0 538418372 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17397] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17397/stat : 17397 (python) S 17337 17339 16913 0 -1 4202560 31 937 0 0 0 0 0 0 18 0 17 0 538443354 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17444] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17444/stat : 17444 (python) S 17337 17339 16913 0 -1 4202560 44 937 0 0 0 0 0 0 18 0 17 0 538445038 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17467] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17467/stat : 17467 (python) S 17337 17339 16913 0 -1 4202560 46 937 0 0 0 0 0 0 18 0 17 0 538458867 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17471] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17471/stat : 17471 (python) S 17337 17339 16913 0 -1 4202560 87 937 0 0 0 0 0 0 18 0 17 0 538459263 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17527] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17527/stat : 17527 (python) S 17337 17339 16913 0 -1 4202560 26 937 0 0 0 0 0 0 20 0 17 0 538466619 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17548] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17548/stat : 17548 (python) S 17337 17339 16913 0 -1 4202560 35 937 0 0 0 0 0 0 18 0 17 0 538489875 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17552] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17552/stat : 17552 (python) S 17337 17339 16913 0 -1 4202560 30 937 0 0 0 0 0 0 18 0 17 0 538494081 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17556] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17556/stat : 17556 (python) S 17337 17339 16913 0 -1 4202560 26 937 0 0 0 0 0 0 20 0 17 0 538498683 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17662] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17662/stat : 17662 (python) S 17337 17339 16913 0 -1 4202560 27 937 0 0 0 0 0 0 20 0 17 0 538522432 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17681] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17681/stat : 17681 (python) S 17337 17339 16913 0 -1 4202560 31 937 0 0 0 0 0 0 20 0 17 0 538540117 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17686] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17686/stat : 17686 (python) S 17337 17339 16913 0 -1 4202560 26 937 0 0 0 0 0 0 20 0 17 0 538543321 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
[pid=17339/tid=17693] ppid=17337 vsize=450024 CPUtime=0 cores=0,2,4,6
/proc/17339/task/17693/stat : 17693 (python) S 17337 17339 16913 0 -1 4202560 29 937 0 0 0 0 0 0 20 0 17 0 538547238 460824576 9008 33554432000 4194304 5683644 140734634892432 18446744073709551615 228256172465 0 0 16781312 2 18446604441721023488 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4026.86
Current children cumulated vsize (KiB) 11844340

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 17339 and gives
#  childrusage.ru_utime.tv_sec=4
#  childrusage.ru_utime.tv_usec=4391
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=173973
# CPU time returned by wait4() is 4.17836
# while last known CPU time is 4026.86
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1300.1
CPU time (s): 4026.86
CPU user time (s): 4022.42
CPU system time (s): 4.44
CPU usage (%): 309.734
Max. virtual memory (cumulated for all children) (KiB): 11844340

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4.00439
system time used= 0.173973
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 30913
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2955
involuntary context switches= 80

runsolver used 4.97724 second user time and 13.7749 second system time

The end

Launcher Data 

Begin job on node136 at 2011-05-01 18:43:20
IDJOB=3317208
IDBENCH=82673
IDSOLVER=1705
FILE ID=node136/3317208-1304268200
RUNJOBID= node136-1304267281-16931
PBS_JOBID= 13173943
Free space on /tmp= 73572 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v15000-c63000-S594553190-015-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317208-1304268200/watcher-3317208-1304268200 -o /tmp/evaluation-result-3317208-1304268200/solver-3317208-1304268200 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317208-1304268200.cnf -seed 1544105914 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 0a98d6d9a7bdb797305aed69a94f9a69
RANDOM SEED=1544105914

node136.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30206460 kB
Buffers:        644536 kB
Cached:        1199400 kB
SwapCached:          0 kB
Active:         779376 kB
Inactive:      1519912 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30206460 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:           33128 kB
Writeback:           0 kB
AnonPages:      454976 kB
Mapped:          23876 kB
Slab:           367400 kB
PageTables:      17700 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 24332168 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73532 MiB
End job on node136 at 2011-05-01 19:05:03