Trace number 3378168

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? 4706.81 1196.21

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v50000-c210000-S746725327-020-UNKNOWN.cnf
MD5SUM0ad997eea0def3552b6d7e45aac8ca7c
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables49999
Number of clauses210000
Sum of the clauses size630000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3210000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.39/0.61	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.39/0.61	c INFO: solving HOME/instance-3378168-1305463314.cnf
0.39/0.61	c INFO: computing task features
1.09/1.36	c INFO: computed variable-clause graph statistics
1.09/1.39	c INFO: computed clause balance statistics
4.78/5.09	c INFO: computed variable balance statistics
23.86/16.39	c INFO: computed small-clause counts
23.86/16.39	c INFO: computed Horn-clause counts
23.86/16.39	c INFO: computed variable graph statistics
23.86/16.39	c INFO: collected features for HOME/instance-3378168-1305463314.cnf in 11.03 s
23.86/16.39	c INFO: running hybridGM3@0 for 97 with 1188 remaining (b = 0.33)
23.86/16.39	c INFO: running hybridGM3@0 for 97 with 1188 remaining (b = 0.21)
23.86/16.39	c INFO: running hybridGM3@0 for 97 with 1188 remaining (b = 0.14)
23.86/16.39	c INFO: running hybridGM3@0 for 194 with 1187 remaining (b = 0.17)
399.47/112.83	c INFO: running hybridGM3@0 for 97 with 1087 remaining (b = 0.08)
399.86/112.99	c INFO: running hybridGM3@0 for 97 with 1087 remaining (b = 0.08)
424.60/119.18	c INFO: running hybridGM3@0 for 97 with 1081 remaining (b = 0.08)
779.37/208.11	c INFO: running hybridGM3@98 for 97 with 992 remaining (b = 0.08)
803.68/214.28	c INFO: running TNM@0 for 97 with 986 remaining (b = 0.08)
804.36/214.46	c INFO: running hybridGM3@100 for 97 with 986 remaining (b = 0.07)
807.64/215.28	c INFO: running hybridGM3@95 for 97 with 985 remaining (b = 0.07)
1183.17/309.41	c INFO: running hybridGM3@100 for 97 with 891 remaining (b = 0.07)
1184.34/309.74	c INFO: running hybridGM3@0 for 97 with 890 remaining (b = 0.06)
1203.46/314.59	c INFO: running hybridGM3@0 for 194 with 886 remaining (b = 0.09)
1211.41/316.58	c INFO: running hybridGM3@0 for 194 with 884 remaining (b = 0.07)
1570.05/406.72	c INFO: running hybridGM3@0 for 194 with 793 remaining (b = 0.05)
1584.36/410.31	c INFO: running hybridGM3@100 for 97 with 790 remaining (b = 0.02)
1986.38/511.44	c INFO: running hybridGM3@101 for 97 with 689 remaining (b = 0.02)
1987.17/511.63	c INFO: running hybridGM3@295 for 97 with 688 remaining (b = 0.02)
2018.20/519.44	c INFO: running hybridGM3@97 for 97 with 681 remaining (b = 0.02)
2352.59/603.54	c INFO: running hybridGM3@99 for 97 with 597 remaining (b = 0.02)
2365.32/606.74	c INFO: running hybridGM3@296 for 97 with 593 remaining (b = 0.02)
2397.11/614.74	c INFO: running TNM@0 for 97 with 585 remaining (b = 0.02)
2397.76/614.96	c INFO: running hybridGM3@294 for 97 with 585 remaining (b = 0.02)
2752.19/704.06	c INFO: running hybridGM3@0 for 194 with 496 remaining (b = 0.04)
2755.34/704.85	c INFO: running TNM@94 for 495 with 495 remaining (b = 0.08)
2795.91/715.06	c INFO: running TNM@0 for 97 with 485 remaining (b = 0.02)
2800.66/716.28	c INFO: running hybridGM3@0 for 194 with 484 remaining (b = 0.03)
3194.96/815.40	c INFO: running TNM@0 for 97 with 385 remaining (b = 0.01)
3534.78/900.94	c INFO: running hybridGM3@100 for 97 with 299 remaining (b = 0.01)
3593.66/915.74	c INFO: running hybridGM3@391 for 97 with 284 remaining (b = 0.01)
3607.50/919.21	c INFO: running hybridGM3@98 for 97 with 281 remaining (b = 0.01)
3937.23/1002.28	c INFO: running hybridGM3@0 for 198 with 198 remaining (b = 0.02)
3980.03/1013.08	c INFO: running TNM@0 for 187 with 187 remaining (b = 0.01)
3985.98/1014.52	c INFO: running hybridGM3@391 for 186 with 186 remaining (b = 0.01)
4706.16/1196.18	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378168-1305463314/watcher-3378168-1305463314 -o /tmp/evaluation-result-3378168-1305463314/solver-3378168-1305463314 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378168-1305463314.cnf -seed 880327032 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.72 3.98 3.89 2/379 27319
/proc/meminfo: memFree=20811404/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=15776 CPUtime=0 cores=1,3,5,7
/proc/27319/stat : 27319 (python) R 27317 27319 27281 0 -1 4202496 935 0 0 0 0 0 0 0 17 0 1 0 657931117 16154624 814 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647142245 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/27319/statm: 3944 814 356 364 0 431 0

[startup+0.0845491 s]
/proc/loadavg: 3.72 3.98 3.89 2/379 27319
/proc/meminfo: memFree=20811404/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=85620 CPUtime=0.06 cores=1,3,5,7
/proc/27319/stat : 27319 (python) R 27317 27319 27281 0 -1 4202496 2629 0 0 0 5 1 0 0 17 0 1 0 657931117 87674880 2348 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647144448 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/27319/statm: 21405 2348 713 364 0 1622 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85620

[startup+0.100514 s]
/proc/loadavg: 3.72 3.98 3.89 2/379 27319
/proc/meminfo: memFree=20811404/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=102012 CPUtime=0.07 cores=1,3,5,7
/proc/27319/stat : 27319 (python) R 27317 27319 27281 0 -1 4202496 3005 0 0 0 5 2 0 0 17 0 1 0 657931117 104460288 2698 33554432000 4194304 5683644 140737406914704 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/27319/statm: 25503 2701 841 364 0 1803 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 102012

[startup+0.3005 s]
/proc/loadavg: 3.72 3.98 3.89 2/379 27319
/proc/meminfo: memFree=20811404/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=135748 CPUtime=0.21 cores=1,3,5,7
/proc/27319/stat : 27319 (python) S 27317 27319 27281 0 -1 4202496 5964 933 1 0 15 6 0 0 17 0 1 0 657931117 139005952 4151 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647142405 0 2147483391 16781312 2 18446744071699109054 0 0 17 1 0 0 0
/proc/27319/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 135748

[startup+0.700417 s]
/proc/loadavg: 3.72 3.98 3.89 2/379 27319
/proc/meminfo: memFree=20811404/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=287396 CPUtime=0.49 cores=1,3,5,7
/proc/27319/stat : 27319 (python) R 27317 27319 27281 0 -1 4202496 11483 933 1 0 36 13 0 0 16 0 1 0 657931117 294293504 9167 33554432000 4194304 5683644 140737406914704 18446744073709551615 210646843600 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/27319/statm: 71849 9167 1761 364 0 7773 0
Current children cumulated CPU time (s) 0.49
Current children cumulated vsize (KiB) 287396

[startup+1.50029 s]
/proc/loadavg: 3.72 3.98 3.89 3/380 27328
/proc/meminfo: memFree=20722736/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=328000 CPUtime=1.28 cores=1,3,5,7
/proc/27319/stat : 27319 (python) R 27317 27319 27281 0 -1 4202496 27451 933 1 0 113 15 0 0 17 0 1 0 657931117 335872000 19385 33554432000 4194304 5683644 140737406914704 18446744073709551615 4539162 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/27319/statm: 82000 19385 1826 364 0 17924 0
Current children cumulated CPU time (s) 1.28
Current children cumulated vsize (KiB) 328000

[startup+3.10102 s]
/proc/loadavg: 3.72 3.98 3.89 3/380 27328
/proc/meminfo: memFree=20699796/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=328000 CPUtime=2.88 cores=1,3,5,7
/proc/27319/stat : 27319 (python) R 27317 27319 27281 0 -1 4202496 27451 933 1 0 273 15 0 0 19 0 1 0 657931117 335872000 19385 33554432000 4194304 5683644 140737406914704 18446744073709551615 47877280638026 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/27319/statm: 82000 19385 1826 364 0 17924 0
Current children cumulated CPU time (s) 2.88
Current children cumulated vsize (KiB) 328000

[startup+6.30048 s]
/proc/loadavg: 3.58 3.95 3.88 3/380 27328
/proc/meminfo: memFree=20699036/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=328000 CPUtime=6.08 cores=1,3,5,7
/proc/27319/stat : 27319 (python) R 27317 27319 27281 0 -1 4202496 27452 933 1 0 593 15 0 0 25 0 1 0 657931117 335872000 19386 33554432000 4194304 5683644 140737406914704 18446744073709551615 4940555 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/27319/statm: 82000 19386 1827 364 0 17924 0
Current children cumulated CPU time (s) 6.08
Current children cumulated vsize (KiB) 328000

[startup+15.7918 s]
/proc/loadavg: 3.58 3.95 3.88 3/380 27328
/proc/meminfo: memFree=20699036/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=365132 CPUtime=11.8 cores=1,3,5,7
/proc/27319/stat : 27319 (python) S 27317 27319 27281 0 -1 4202496 30214 933 1 0 1142 38 0 0 25 0 5 0 657931117 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/27319/statm: 91283 18480 1881 364 0 27192 0
Current children cumulated CPU time (s) 11.8
Current children cumulated vsize (KiB) 365132
heavy processes:

[startup+25.5012 s]
/proc/loadavg: 6.03 4.48 4.05 10/404 27352
/proc/meminfo: memFree=20413348/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=365132 CPUtime=11.8 cores=1,3,5,7
/proc/27319/stat : 27319 (python) S 27317 27319 27281 0 -1 4202496 30214 933 1 0 1142 38 0 0 25 0 5 0 657931117 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/27319/statm: 91283 18480 1881 364 0 27192 0
[pid=27341] ppid=27319 vsize=324120 CPUtime=0.02 cores=1,3,5,7
/proc/27341/stat : 27341 (python) S 27319 27319 27281 0 -1 4202560 891 0 0 0 1 1 0 0 15 0 1 0 657932356 331898880 16959 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166479 0 0 16781312 514 5278482646068855048 0 0 17 7 0 0 0
/proc/27341/statm: 81030 16959 376 364 0 16948 0
[pid=27343] ppid=27341 vsize=276240 CPUtime=13.07 cores=1,3,5,7
/proc/27343/stat : 27343 (hybridGM3) R 27341 27343 27343 0 -1 4202496 9512 0 0 0 1306 1 0 0 25 0 1 0 657932358 282869760 9249 33554432000 134512640 135204385 4288587072 18446744073709551615 134537272 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/27343/statm: 69060 9249 64 169 0 68890 0
[pid=27344] ppid=27319 vsize=334376 CPUtime=0.03 cores=1,3,5,7
/proc/27344/stat : 27344 (python) S 27319 27319 27281 0 -1 4202560 913 0 0 0 1 2 0 0 15 0 1 0 657932361 342401024 16957 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166534 0 0 16781312 514 5278482646068855044 0 0 17 5 0 0 0
/proc/27344/statm: 83594 16957 370 364 0 19509 0
[pid=27346] ppid=27344 vsize=276240 CPUtime=12.73 cores=1,3,5,7
/proc/27346/stat : 27346 (hybridGM3) R 27344 27346 27346 0 -1 4202496 9515 0 0 0 1272 1 0 0 25 0 1 0 657932374 282869760 9253 33554432000 134512640 135204385 4291242864 18446744073709551615 134536767 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/27346/statm: 69060 9253 64 169 0 68890 0
[pid=27347] ppid=27319 vsize=344632 CPUtime=0.02 cores=1,3,5,7
/proc/27347/stat : 27347 (python) S 27319 27319 27281 0 -1 4202560 900 0 0 0 1 1 0 0 15 0 1 0 657932389 352903168 16961 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166534 0 0 16781312 514 5278482646068855045 0 0 17 5 0 0 0
/proc/27347/statm: 86158 16961 370 364 0 22070 0
[pid=27351] ppid=27347 vsize=276240 CPUtime=7.79 cores=1,3,5,7
/proc/27351/stat : 27351 (hybridGM3) R 27347 27351 27351 0 -1 4202496 9522 0 0 0 778 1 0 0 25 0 1 0 657932391 282869760 9261 33554432000 134512640 135204385 4293734992 18446744073709551615 134545851 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/27351/statm: 69060 9261 64 169 0 68890 0
[pid=27349] ppid=27319 vsize=354888 CPUtime=0.02 cores=1,3,5,7
/proc/27349/stat : 27349 (python) S 27319 27319 27281 0 -1 4202560 913 0 0 0 1 1 0 0 15 0 1 0 657932390 363405312 16965 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166534 0 0 16781312 514 5278482646068855047 0 0 17 5 0 0 0
/proc/27349/statm: 88722 16965 370 364 0 24631 0
[pid=27352] ppid=27349 vsize=276236 CPUtime=6.16 cores=1,3,5,7
/proc/27352/stat : 27352 (hybridGM3) R 27349 27352 27352 0 -1 4202496 9521 0 0 0 553 63 0 0 25 0 1 0 657932391 282865664 9262 33554432000 134512640 135204385 4289281648 18446744073709551615 134537272 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/27352/statm: 69059 9262 64 169 0 68889 0
[pid=27319/tid=27342] ppid=27317 vsize=365132 CPUtime=0.04 cores=1,3,5,7
/proc/27319/task/27342/stat : 27342 (python) S 27317 27319 27281 0 -1 4202560 30 933 0 0 1 3 0 0 25 0 5 0 657932356 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 1 0 0 0
[pid=27319/tid=27345] ppid=27317 vsize=365132 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27345/stat : 27345 (python) S 27317 27319 27281 0 -1 4202560 22 933 0 0 0 0 0 0 25 0 5 0 657932364 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27348] ppid=27317 vsize=365132 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27348/stat : 27348 (python) S 27317 27319 27281 0 -1 4202560 40 933 0 0 0 0 0 0 25 0 5 0 657932389 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27350] ppid=27317 vsize=365132 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27350/stat : 27350 (python) S 27317 27319 27281 0 -1 4202560 18 933 0 0 0 0 0 0 25 0 5 0 657932390 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 51.64
Current children cumulated vsize (KiB) 2828104

[startup+51.1159 s]
/proc/loadavg: 6.76 4.78 4.16 9/404 27353
/proc/meminfo: memFree=20414856/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=365132 CPUtime=11.8 cores=1,3,5,7
/proc/27319/stat : 27319 (python) S 27317 27319 27281 0 -1 4202496 30214 933 1 0 1142 38 0 0 25 0 5 0 657931117 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/27319/statm: 91283 18480 1881 364 0 27192 0
[pid=27341] ppid=27319 vsize=324120 CPUtime=0.1 cores=1,3,5,7
/proc/27341/stat : 27341 (python) S 27319 27319 27281 0 -1 4202560 901 0 0 0 4 6 0 0 15 0 1 0 657932356 331898880 16959 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166479 0 0 16781312 514 5278482646068855048 0 0 17 7 0 0 0
/proc/27341/statm: 81030 16959 376 364 0 16948 0
[pid=27343] ppid=27341 vsize=276240 CPUtime=38.68 cores=1,3,5,7
/proc/27343/stat : 27343 (hybridGM3) R 27341 27343 27343 0 -1 4202496 9515 0 0 0 3867 1 0 0 25 0 1 0 657932358 282869760 9252 33554432000 134512640 135204385 4288587072 18446744073709551615 134537292 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/27343/statm: 69060 9252 64 169 0 68890 0
[pid=27344] ppid=27319 vsize=334376 CPUtime=0.1 cores=1,3,5,7
/proc/27344/stat : 27344 (python) S 27319 27319 27281 0 -1 4202560 924 0 0 0 4 6 0 0 15 0 1 0 657932361 342401024 16957 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166534 0 0 16781312 514 5278482646068855044 0 0 17 5 0 0 0
/proc/27344/statm: 83594 16957 370 364 0 19509 0
[pid=27346] ppid=27344 vsize=276240 CPUtime=37.91 cores=1,3,5,7
/proc/27346/stat : 27346 (hybridGM3) R 27344 27346 27346 0 -1 4202496 9515 0 0 0 3790 1 0 0 25 0 1 0 657932374 282869760 9253 33554432000 134512640 135204385 4291242864 18446744073709551615 134545851 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/27346/statm: 69060 9253 64 169 0 68890 0
[pid=27347] ppid=27319 vsize=344632 CPUtime=0.1 cores=1,3,5,7
/proc/27347/stat : 27347 (python) S 27319 27319 27281 0 -1 4202560 909 0 0 0 4 6 0 0 15 0 1 0 657932389 352903168 16961 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166534 0 0 16781312 514 5278482646068855045 0 0 17 5 0 0 0
/proc/27347/statm: 86158 16961 370 364 0 22070 0
[pid=27351] ppid=27347 vsize=276240 CPUtime=33.17 cores=1,3,5,7
/proc/27351/stat : 27351 (hybridGM3) R 27347 27351 27351 0 -1 4202496 9522 0 0 0 3316 1 0 0 25 0 1 0 657932391 282869760 9261 33554432000 134512640 135204385 4293734992 18446744073709551615 134536663 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/27351/statm: 69060 9261 64 169 0 68890 0
[pid=27349] ppid=27319 vsize=354888 CPUtime=0.1 cores=1,3,5,7
/proc/27349/stat : 27349 (python) S 27319 27319 27281 0 -1 4202560 922 0 0 0 6 4 0 0 15 0 1 0 657932390 363405312 16965 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166534 0 0 16781312 514 5278482646068855047 0 0 17 5 0 0 0
/proc/27349/statm: 88722 16965 370 364 0 24631 0
[pid=27352] ppid=27349 vsize=276236 CPUtime=31.62 cores=1,3,5,7
/proc/27352/stat : 27352 (hybridGM3) R 27349 27352 27352 0 -1 4202496 9521 0 0 0 3072 90 0 0 25 0 1 0 657932391 282865664 9262 33554432000 134512640 135204385 4289281648 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/27352/statm: 69059 9262 64 169 0 68889 0
[pid=27319/tid=27342] ppid=27317 vsize=365132 CPUtime=0.04 cores=1,3,5,7
/proc/27319/task/27342/stat : 27342 (python) S 27317 27319 27281 0 -1 4202560 30 933 0 0 1 3 0 0 25 0 5 0 657932356 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 1 0 0 0
[pid=27319/tid=27345] ppid=27317 vsize=365132 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27345/stat : 27345 (python) S 27317 27319 27281 0 -1 4202560 22 933 0 0 0 0 0 0 25 0 5 0 657932364 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27348] ppid=27317 vsize=365132 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27348/stat : 27348 (python) S 27317 27319 27281 0 -1 4202560 40 933 0 0 0 0 0 0 25 0 5 0 657932389 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27350] ppid=27317 vsize=365132 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27350/stat : 27350 (python) S 27317 27319 27281 0 -1 4202560 18 933 0 0 0 0 0 0 25 0 5 0 657932390 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 153.58
Current children cumulated vsize (KiB) 2828104

[startup+102.301 s]
/proc/loadavg: 7.62 5.32 4.38 9/404 27354
/proc/meminfo: memFree=20413824/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=365132 CPUtime=11.8 cores=1,3,5,7
/proc/27319/stat : 27319 (python) S 27317 27319 27281 0 -1 4202496 30214 933 1 0 1142 38 0 0 25 0 5 0 657931117 373895168 18480 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/27319/statm: 91283 18480 1881 364 0 27192 0
[pid=27341] ppid=27319 vsize=324120 CPUtime=0.26 cores=1,3,5,7
/proc/27341/stat : 27341 (python) S 27319 27319 27281 0 -1 4202560 920 0 0 0 13 13 0 0 15 0 1 0 657932356 331898880 16959 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166479 0 0 16781312 514 5278482646068855048 0 0 17 7 0 0 0
/proc/27341/statm: 81030 16959 376 364 0 16948 0
[pid=27343] ppid=27341 vsize=276240 CPUtime=89.87 cores=1,3,5,7
/proc/27343/stat : 27343 (hybridGM3) R 27341 27343 27343 0 -1 4202496 9515 0 0 0 8986 1 0 0 25 0 1 0 657932358 282869760 9252 33554432000 134512640 135204385 4288587072 18446744073709551615 134536784 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/27343/statm: 69060 9252 64 169 0 68890 0
[pid=27344] ppid=27319 vsize=334376 CPUtime=0.27 cores=1,3,5,7
/proc/27344/stat : 27344 (python) S 27319 27319 27281 0 -1 4202560 941 0 0 0 14 13 0 0 15 0 1 0 657932361 342401024 16957 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166534 0 0 16781312 514 5278482646068855044 0 0 17 5 0 0 0
/proc/27344/statm: 83594 16957 370 364 0 19509 0
[pid=27346] ppid=27344 vsize=276240 CPUtime=88.25 cores=1,3,5,7
/proc/27346/stat : 27346 (hybridGM3) R 27344 27346 27346 0 -1 4202496 9515 0 0 0 8824 1 0 0 25 0 1 0 657932374 282869760 9253 33554432000 134512640 135204385 4291242864 18446744073709551615 134549044 0 0 16781312 0 0 0 0 17 7 0 0 0

################
# More data... #
################

/proc/27319/task/27471/stat : 27471 (python) S 27317 27319 27281 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 17 0 658031345 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27475] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27475/stat : 27475 (python) S 27317 27319 27281 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 17 0 658032426 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4705.73
Current children cumulated vsize (KiB) 12309296

[startup+1196.1 s]
/proc/loadavg: 8.21 8.15 7.02 10/504 27493
/proc/meminfo: memFree=19584688/32951132 swapFree=64495980/67111528
[pid=27319] ppid=27317 vsize=488204 CPUtime=12.12 cores=1,3,5,7
/proc/27319/stat : 27319 (python) S 27317 27319 27281 0 -1 4202496 42251 933 1 0 1168 44 0 0 18 0 17 0 657931117 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/27319/statm: 122051 18553 1906 364 0 57924 0
[pid=27341] ppid=27319 vsize=334364 CPUtime=0.89 cores=1,3,5,7
/proc/27341/stat : 27341 (python) S 27319 27319 27281 0 -1 4202560 1225 0 0 0 46 43 0 0 15 0 2 0 657932356 342388736 17004 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/27341/statm: 83591 17004 416 364 0 19509 0
[pid=27343] ppid=27341 vsize=276240 CPUtime=296 cores=1,3,5,7
/proc/27343/stat : 27343 (hybridGM3) T 27341 27343 27343 0 -1 4202496 9519 0 0 0 29599 1 0 0 25 0 1 0 657932358 282869760 9256 33554432000 134512640 135204385 4288587072 18446744073709551615 134537510 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/27343/statm: 69060 9256 64 169 0 68890 0
[pid=27341/tid=27359] ppid=27319 vsize=334364 CPUtime=0 cores=1,3,5,7
/proc/27341/task/27359/stat : 27359 (python) S 27319 27319 27281 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 657942399 342388736 17004 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604466419335616 0 0 -1 5 0 0 0
[pid=27344] ppid=27319 vsize=334376 CPUtime=1.24 cores=1,3,5,7
/proc/27344/stat : 27344 (python) S 27319 27319 27281 0 -1 4202560 1142 0 0 0 65 59 0 0 15 0 2 0 657932361 342401024 16987 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27344/statm: 83594 16987 399 364 0 19509 0
[pid=27346] ppid=27344 vsize=276240 CPUtime=392.97 cores=1,3,5,7
/proc/27346/stat : 27346 (hybridGM3) T 27344 27346 27346 0 -1 4202496 9518 0 0 0 39296 1 0 0 25 0 1 0 657932374 282869760 9256 33554432000 134512640 135204385 4291242864 18446744073709551615 134537410 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/27346/statm: 69060 9256 64 169 0 68890 0
[pid=27344/tid=27360] ppid=27319 vsize=334376 CPUtime=0 cores=1,3,5,7
/proc/27344/task/27360/stat : 27360 (python) S 27319 27319 27281 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 657942415 342401024 16987 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604466681128128 0 0 -1 5 0 0 0
[pid=27347] ppid=27319 vsize=344632 CPUtime=0.91 cores=1,3,5,7
/proc/27347/stat : 27347 (python) S 27319 27319 27281 0 -1 4202560 1133 0 0 0 47 44 0 0 15 0 2 0 657932389 352903168 16991 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27347/statm: 86158 16991 399 364 0 22070 0
[pid=27351] ppid=27347 vsize=276240 CPUtime=296.4 cores=1,3,5,7
/proc/27351/stat : 27351 (hybridGM3) T 27347 27351 27351 0 -1 4202496 9522 0 0 0 29639 1 0 0 25 0 1 0 657932391 282869760 9261 33554432000 134512640 135204385 4293734992 18446744073709551615 134537020 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/27351/statm: 69060 9261 64 169 0 68890 0
[pid=27347/tid=27361] ppid=27319 vsize=344632 CPUtime=0 cores=1,3,5,7
/proc/27347/task/27361/stat : 27361 (python) S 27319 27319 27281 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 657943035 352903168 16991 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604454225327232 0 0 -1 5 0 0 0
[pid=27349] ppid=27319 vsize=354888 CPUtime=0.93 cores=1,3,5,7
/proc/27349/stat : 27349 (python) S 27319 27319 27281 0 -1 4202560 1147 0 0 0 51 42 0 0 16 0 2 0 657932390 363405312 16995 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27349/statm: 88722 16995 399 364 0 24631 0
[pid=27352] ppid=27349 vsize=276236 CPUtime=293.07 cores=1,3,5,7
/proc/27352/stat : 27352 (hybridGM3) T 27349 27352 27352 0 -1 4202496 9521 0 0 0 29215 92 0 0 25 0 1 0 657932391 282865664 9262 33554432000 134512640 135204385 4289281648 18446744073709551615 134546153 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/27352/statm: 69059 9262 64 169 0 68889 0
[pid=27349/tid=27369] ppid=27319 vsize=354888 CPUtime=0 cores=1,3,5,7
/proc/27349/task/27369/stat : 27369 (python) S 27319 27319 27281 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 657952644 363405312 16995 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604454225328128 0 0 -1 5 0 0 0
[pid=27366] ppid=27319 vsize=365144 CPUtime=1.94 cores=1,3,5,7
/proc/27366/stat : 27366 (python) S 27319 27319 27281 0 -1 4202560 1147 0 0 0 87 107 0 0 18 0 2 0 657952546 373907456 16999 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166534 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27366/statm: 91286 16999 399 364 0 27192 0
[pid=27368] ppid=27366 vsize=471116 CPUtime=588.76 cores=1,3,5,7
/proc/27368/stat : 27368 (TNM) R 27366 27368 27368 0 -1 4202496 8431 0 0 0 58875 1 0 0 25 0 1 0 657952546 482422784 8172 33554432000 134512640 135008748 4289646576 18446744073709551615 134527840 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/27368/statm: 117779 8172 46 122 0 116679 0
[pid=27366/tid=27387] ppid=27319 vsize=365144 CPUtime=0 cores=1,3,5,7
/proc/27366/task/27387/stat : 27387 (python) S 27319 27319 27281 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657962576 373907456 16999 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604468779624640 0 0 -1 7 0 0 0
[pid=27384] ppid=27319 vsize=375400 CPUtime=1.31 cores=1,3,5,7
/proc/27384/stat : 27384 (python) S 27319 27319 27281 0 -1 4202560 1162 0 0 0 66 65 0 0 16 0 2 0 657962092 384409600 17003 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27384/statm: 93850 17003 399 364 0 29753 0
[pid=27386] ppid=27384 vsize=276240 CPUtime=393.95 cores=1,3,5,7
/proc/27386/stat : 27386 (hybridGM3) T 27384 27386 27386 0 -1 4202496 9522 0 0 0 39392 3 0 0 25 0 1 0 657962092 282869760 9262 33554432000 134512640 135204385 4287493728 18446744073709551615 134537401 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/27386/statm: 69060 9262 64 169 0 68890 0
[pid=27384/tid=27406] ppid=27319 vsize=375400 CPUtime=0 cores=1,3,5,7
/proc/27384/task/27406/stat : 27406 (python) S 27319 27319 27281 0 -1 4202560 51 0 0 0 0 0 0 0 15 0 2 0 657972147 384409600 17003 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604466681127232 0 0 -1 5 0 0 0
[pid=27388] ppid=27319 vsize=385656 CPUtime=0.92 cores=1,3,5,7
/proc/27388/stat : 27388 (python) S 27319 27319 27281 0 -1 4202560 1155 0 0 0 43 49 0 0 15 0 2 0 657962577 394911744 17007 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27388/statm: 96414 17007 399 364 0 32314 0
[pid=27390] ppid=27388 vsize=276236 CPUtime=292.05 cores=1,3,5,7
/proc/27390/stat : 27390 (hybridGM3) T 27388 27390 27390 0 -1 4202496 9517 0 0 0 29204 1 0 0 25 0 1 0 657962577 282865664 9254 33554432000 134512640 135204385 4291795968 18446744073709551615 134546082 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/27390/statm: 69059 9254 64 169 0 68889 0
[pid=27388/tid=27414] ppid=27319 vsize=385656 CPUtime=0 cores=1,3,5,7
/proc/27388/task/27414/stat : 27414 (python) S 27319 27319 27281 0 -1 4202560 50 0 0 0 0 0 0 0 15 0 2 0 657982261 394911744 17007 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604466681129024 0 0 -1 5 0 0 0
[pid=27391] ppid=27319 vsize=395916 CPUtime=1.89 cores=1,3,5,7
/proc/27391/stat : 27391 (python) R 27319 27319 27281 0 -1 4202560 1186 0 0 0 88 101 0 0 16 0 2 0 657962775 405417984 17017 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647146299 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27391/statm: 98978 17017 405 364 0 34875 0
[pid=27393] ppid=27391 vsize=276244 CPUtime=575.18 cores=1,3,5,7
/proc/27393/stat : 27393 (hybridGM3) R 27391 27393 27393 0 -1 4202496 9528 0 0 0 57516 2 0 0 25 0 1 0 657962775 282873856 9258 33554432000 134512640 135204385 4287973024 18446744073709551615 134537583 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/27393/statm: 69061 9258 64 169 0 68891 0
[pid=27391/tid=27415] ppid=27319 vsize=395912 CPUtime=0 cores=1,3,5,7
/proc/27391/task/27415/stat : 27415 (python) S 27319 27319 27281 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 657983060 405413888 17017 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604465041204544 0 0 -1 5 0 0 0
[pid=27403] ppid=27319 vsize=406168 CPUtime=0.94 cores=1,3,5,7
/proc/27403/stat : 27403 (python) S 27319 27319 27281 0 -1 4202560 1197 0 0 0 47 47 0 0 15 0 2 0 657971790 415916032 17021 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27403/statm: 101542 17021 405 364 0 37436 0
[pid=27405] ppid=27403 vsize=276240 CPUtime=296.79 cores=1,3,5,7
/proc/27405/stat : 27405 (hybridGM3) T 27403 27405 27405 0 -1 4202496 9523 0 0 0 29677 2 0 0 25 0 1 0 657971790 282869760 9258 33554432000 134512640 135204385 4291000192 18446744073709551615 134546082 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/27405/statm: 69060 9258 64 169 0 68890 0
[pid=27403/tid=27419] ppid=27319 vsize=406168 CPUtime=0 cores=1,3,5,7
/proc/27403/task/27419/stat : 27419 (python) S 27319 27319 27281 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 657991470 415916032 17021 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604466419337408 0 0 -1 5 0 0 0
[pid=27421] ppid=27319 vsize=416424 CPUtime=0.32 cores=1,3,5,7
/proc/27421/stat : 27421 (python) S 27319 27319 27281 0 -1 4202560 1124 0 0 0 15 17 0 0 19 0 2 0 657992591 426418176 17019 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27421/statm: 104106 17019 399 364 0 39997 0
[pid=27423] ppid=27421 vsize=471116 CPUtime=98.01 cores=1,3,5,7
/proc/27423/stat : 27423 (TNM) T 27421 27423 27423 0 -1 4202496 8433 0 0 0 9799 2 0 0 25 0 1 0 657992591 482422784 8170 33554432000 134512640 135008748 4290434864 18446744073709551615 134527782 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/27423/statm: 117779 8170 46 122 0 116679 0
[pid=27421/tid=27436] ppid=27319 vsize=416424 CPUtime=0 cores=1,3,5,7
/proc/27421/task/27436/stat : 27436 (python) S 27319 27319 27281 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 658002623 426418176 17019 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604455868924288 0 0 -1 5 0 0 0
[pid=27433] ppid=27319 vsize=426680 CPUtime=0.98 cores=1,3,5,7
/proc/27433/stat : 27433 (python) S 27319 27319 27281 0 -1 4202560 1190 0 0 0 47 51 0 0 16 0 2 0 658001523 436920320 17023 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27433/statm: 106670 17023 399 364 0 42558 0
[pid=27435] ppid=27433 vsize=276236 CPUtime=295.58 cores=1,3,5,7
/proc/27435/stat : 27435 (hybridGM3) T 27433 27435 27435 0 -1 4202496 9518 0 0 0 29553 5 0 0 25 0 1 0 658001523 282865664 9254 33554432000 134512640 135204385 4289736416 18446744073709551615 134536744 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/27435/statm: 69059 9254 64 169 0 68889 0
[pid=27433/tid=27460] ppid=27319 vsize=426680 CPUtime=0 cores=1,3,5,7
/proc/27433/task/27460/stat : 27460 (python) S 27319 27319 27281 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 658021210 436920320 17023 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604470807783296 0 0 -1 5 0 0 0
[pid=27437] ppid=27319 vsize=436936 CPUtime=0.32 cores=1,3,5,7
/proc/27437/stat : 27437 (python) S 27319 27319 27281 0 -1 4202560 1131 0 0 0 16 16 0 0 19 0 2 0 658002624 447422464 17027 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27437/statm: 109234 17027 399 364 0 45119 0
[pid=27439] ppid=27437 vsize=471116 CPUtime=98.98 cores=1,3,5,7
/proc/27439/stat : 27439 (TNM) T 27437 27439 27439 0 -1 4202496 8440 0 0 0 9897 1 0 0 25 0 1 0 658002624 482422784 8172 33554432000 134512640 135008748 4294650592 18446744073709551615 134527444 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/27439/statm: 117779 8172 46 122 0 116679 0
[pid=27437/tid=27450] ppid=27319 vsize=436936 CPUtime=0 cores=1,3,5,7
/proc/27437/task/27450/stat : 27450 (python) S 27319 27319 27281 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 658012656 447422464 17027 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604468779625536 0 0 -1 5 0 0 0
[pid=27440] ppid=27319 vsize=447192 CPUtime=0.98 cores=1,3,5,7
/proc/27440/stat : 27440 (python) S 27319 27319 27281 0 -1 4202560 1178 0 0 0 46 52 0 0 16 0 2 0 658002745 457924608 17031 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27440/statm: 111798 17031 399 364 0 47680 0
[pid=27442] ppid=27440 vsize=276236 CPUtime=293.06 cores=1,3,5,7
/proc/27442/stat : 27442 (hybridGM3) T 27440 27442 27442 0 -1 4202496 9523 0 0 0 29304 2 0 0 25 0 1 0 658002746 282865664 9255 33554432000 134512640 135204385 4292399552 18446744073709551615 134537573 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/27442/statm: 69059 9255 64 169 0 68889 0
[pid=27440/tid=27462] ppid=27319 vsize=447192 CPUtime=0 cores=1,3,5,7
/proc/27440/task/27462/stat : 27462 (python) S 27319 27319 27281 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 658023037 457924608 17031 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604466881863936 0 0 -1 5 0 0 0
[pid=27451] ppid=27319 vsize=457448 CPUtime=0.33 cores=1,3,5,7
/proc/27451/stat : 27451 (python) S 27319 27319 27281 0 -1 4202560 1153 0 0 0 16 17 0 0 19 0 2 0 658012657 468426752 17035 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658973131 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/27451/statm: 114362 17035 399 364 0 50241 0
[pid=27453] ppid=27451 vsize=471116 CPUtime=97.76 cores=1,3,5,7
/proc/27453/stat : 27453 (TNM) T 27451 27453 27453 0 -1 4202496 8445 0 0 0 9774 2 0 0 25 0 1 0 658012658 482422784 8171 33554432000 134512640 135008748 4290671616 18446744073709551615 134529176 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/27453/statm: 117779 8171 46 122 0 116679 0
[pid=27451/tid=27461] ppid=27319 vsize=457448 CPUtime=0 cores=1,3,5,7
/proc/27451/task/27461/stat : 27461 (python) S 27319 27319 27281 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 658022691 468426752 17035 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 514 18446604455978932032 0 0 -1 5 0 0 0
[pid=27470] ppid=27319 vsize=467704 CPUtime=0.64 cores=1,3,5,7
/proc/27470/stat : 27470 (python) S 27319 27319 27281 0 -1 4202560 1070 0 0 0 32 32 0 0 15 0 1 0 658031345 478928896 17016 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166534 0 0 16781312 514 5278482646068855046 0 0 17 5 0 0 0
/proc/27470/statm: 116926 17016 377 364 0 52802 0
[pid=27472] ppid=27470 vsize=276240 CPUtime=193.63 cores=1,3,5,7
/proc/27472/stat : 27472 (hybridGM3) R 27470 27472 27472 0 -1 4202496 9528 0 0 0 19361 2 0 0 25 0 1 0 658031345 282869760 9255 33554432000 134512640 135204385 4293676464 18446744073709551615 134537510 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/27472/statm: 69060 9255 64 169 0 68890 0
[pid=27474] ppid=27319 vsize=477960 CPUtime=0.59 cores=1,3,5,7
/proc/27474/stat : 27474 (python) S 27319 27319 27281 0 -1 4202560 1055 0 0 0 26 33 0 0 18 0 1 0 658032425 489431040 17020 33554432000 4194304 5683644 140737406914704 18446744073709551615 210647166534 0 0 16781312 514 5278482646068855044 0 0 17 5 0 0 0
/proc/27474/statm: 119490 17020 377 364 0 55363 0
[pid=27476] ppid=27474 vsize=471116 CPUtime=176.72 cores=1,3,5,7
/proc/27476/stat : 27476 (TNM) R 27474 27476 27476 0 -1 4202496 8445 0 0 0 17670 2 0 0 25 0 1 0 658032426 482422784 8172 33554432000 134512640 135008748 4292138176 18446744073709551615 134527436 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/27476/statm: 117779 8172 46 122 0 116679 0
[pid=27319/tid=27342] ppid=27317 vsize=488204 CPUtime=0.04 cores=1,3,5,7
/proc/27319/task/27342/stat : 27342 (python) S 27317 27319 27281 0 -1 4202560 36 933 0 0 1 3 0 0 18 0 17 0 657932356 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 1 0 0 0
[pid=27319/tid=27345] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27345/stat : 27345 (python) S 27317 27319 27281 0 -1 4202560 36 933 0 0 0 0 0 0 18 0 17 0 657932364 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27348] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27348/stat : 27348 (python) S 27317 27319 27281 0 -1 4202560 44 933 0 0 0 0 0 0 18 0 17 0 657932389 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27350] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27350/stat : 27350 (python) S 27317 27319 27281 0 -1 4202560 21 933 0 0 0 0 0 0 18 0 17 0 657932390 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27367] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27367/stat : 27367 (python) S 27317 27319 27281 0 -1 4202560 32 933 0 0 0 0 0 0 18 0 17 0 657952546 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27385] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27385/stat : 27385 (python) S 27317 27319 27281 0 -1 4202560 39 933 0 0 0 0 0 0 18 0 17 0 657962092 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27389] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27389/stat : 27389 (python) S 27317 27319 27281 0 -1 4202560 34 933 0 0 0 0 0 0 18 0 17 0 657962577 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27392] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27392/stat : 27392 (python) S 27317 27319 27281 0 -1 4202560 41 933 0 0 0 0 0 0 18 0 17 0 657962775 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27404] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27404/stat : 27404 (python) S 27317 27319 27281 0 -1 4202560 23 933 0 0 0 0 0 0 18 0 17 0 657971790 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27422] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27422/stat : 27422 (python) S 27317 27319 27281 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 17 0 657992591 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27434] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27434/stat : 27434 (python) S 27317 27319 27281 0 -1 4202560 33 933 0 0 0 0 0 0 18 0 17 0 658001523 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27438] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27438/stat : 27438 (python) S 27317 27319 27281 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 17 0 658002624 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27441] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27441/stat : 27441 (python) S 27317 27319 27281 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 17 0 658002746 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27452] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27452/stat : 27452 (python) S 27317 27319 27281 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 17 0 658012658 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27471] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27471/stat : 27471 (python) S 27317 27319 27281 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 17 0 658031345 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
[pid=27319/tid=27475] ppid=27317 vsize=488204 CPUtime=0 cores=1,3,5,7
/proc/27319/task/27475/stat : 27475 (python) S 27317 27319 27281 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 17 0 658032426 499920896 18553 33554432000 4194304 5683644 140737406914704 18446744073709551615 210658970033 0 0 16781312 2 18446604458095919168 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4706.16
Current children cumulated vsize (KiB) 12309300

Child status: 0
Real time (s): 1196.21
CPU time (s): 4706.81
CPU user time (s): 4697.15
CPU system time (s): 9.65853
CPU usage (%): 393.476
Max. virtual memory (cumulated for all children) (KiB): 12309300

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4697.15
system time used= 9.65853
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 218288
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 13153
involuntary context switches= 29612

runsolver used 7.55285 second user time and 20.1259 second system time

The end

Launcher Data 

Begin job on node129 at 2011-05-15 14:41:55
IDJOB=3378168
IDBENCH=82577
IDSOLVER=1859
FILE ID=node129/3378168-1305463314
RUNJOBID= node129-1305463314-27300
PBS_JOBID= 13324993
Free space on /tmp= 73356 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v50000-c210000-S746725327-020-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378168-1305463314/watcher-3378168-1305463314 -o /tmp/evaluation-result-3378168-1305463314/solver-3378168-1305463314 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378168-1305463314.cnf -seed 880327032 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 0ad997eea0def3552b6d7e45aac8ca7c
RANDOM SEED=880327032

node129.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951132 kB
MemFree:      20811808 kB
Buffers:        265912 kB
Cached:        6415708 kB
SwapCached:      52668 kB
Active:        3381680 kB
Inactive:      8516336 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951132 kB
LowFree:      20811808 kB
SwapTotal:    67111528 kB
SwapFree:     64495980 kB
Dirty:           69876 kB
Writeback:          24 kB
AnonPages:     5172100 kB
Mapped:          21668 kB
Slab:           134464 kB
PageTables:      40460 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587092 kB
Committed_AS: 93155764 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296000 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73324 MiB
End job on node129 at 2011-05-15 15:01:51