Trace number 3378263

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? 4729.73 1197.61

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v50000-c210000-S517749870-003-UNKNOWN.cnf
MD5SUM4033245e3bd21ace8cfcd57089ff73c5
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark213.183
Satisfiable
(Un)Satisfiability was proved
Number of variables50000
Number of clauses210000
Sum of the clauses size630000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3210000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.35/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.67	c INFO: solving HOME/instance-3378263-1305464356.cnf
0.35/0.67	c INFO: computing task features
1.13/1.41	c INFO: computed variable-clause graph statistics
1.13/1.44	c INFO: computed clause balance statistics
4.83/5.16	c INFO: computed variable balance statistics
7.93/8.28	c INFO: computed small-clause counts
7.93/8.29	c INFO: computed Horn-clause counts
11.03/11.38	c INFO: computed variable graph statistics
11.03/11.38	c INFO: collected features for HOME/instance-3378263-1305464356.cnf in 10.68 s
11.03/11.39	c INFO: running hybridGM3@0 for 97 with 1189 remaining (b = 0.35)
11.13/11.40	c INFO: running hybridGM3@0 for 97 with 1189 remaining (b = 0.21)
11.13/11.41	c INFO: running hybridGM3@0 for 97 with 1189 remaining (b = 0.14)
11.13/11.42	c INFO: running hybridGM3@0 for 194 with 1189 remaining (b = 0.18)
407.37/110.82	c INFO: running hybridGM3@0 for 97 with 1089 remaining (b = 0.08)
407.37/110.83	c INFO: running hybridGM3@0 for 97 with 1089 remaining (b = 0.08)
407.37/110.87	c INFO: running hybridGM3@0 for 97 with 1089 remaining (b = 0.08)
786.89/206.08	c INFO: running hybridGM3@99 for 97 with 994 remaining (b = 0.08)
791.28/207.10	c INFO: running TNM@0 for 97 with 993 remaining (b = 0.08)
803.29/210.18	c INFO: running hybridGM3@99 for 97 with 990 remaining (b = 0.07)
807.24/211.19	c INFO: running hybridGM3@97 for 97 with 989 remaining (b = 0.07)
1190.78/307.36	c INFO: running hybridGM3@95 for 97 with 893 remaining (b = 0.07)
1190.78/307.39	c INFO: running hybridGM3@0 for 97 with 893 remaining (b = 0.06)
1195.17/308.47	c INFO: running hybridGM3@0 for 194 with 892 remaining (b = 0.09)
1203.11/310.49	c INFO: running hybridGM3@0 for 194 with 890 remaining (b = 0.07)
1573.82/403.66	c INFO: running hybridGM3@0 for 194 with 797 remaining (b = 0.05)
1590.47/407.89	c INFO: running hybridGM3@97 for 97 with 792 remaining (b = 0.02)
1969.50/503.17	c INFO: running hybridGM3@192 for 97 with 697 remaining (b = 0.02)
1985.80/507.24	c INFO: running hybridGM3@95 for 97 with 693 remaining (b = 0.02)
2001.73/511.24	c INFO: running hybridGM3@97 for 97 with 689 remaining (b = 0.02)
2364.86/602.52	c INFO: running hybridGM3@100 for 97 with 598 remaining (b = 0.02)
2372.40/604.48	c INFO: running hybridGM3@294 for 97 with 596 remaining (b = 0.02)
2376.75/605.52	c INFO: running hybridGM3@296 for 97 with 595 remaining (b = 0.02)
2388.68/608.54	c INFO: running hybridGM3@293 for 97 with 592 remaining (b = 0.02)
2747.83/698.85	c INFO: running TNM@97 for 501 with 501 remaining (b = 0.10)
2759.39/701.77	c INFO: running TNM@0 for 97 with 498 remaining (b = 0.05)
2767.70/703.82	c INFO: running TNM@0 for 97 with 496 remaining (b = 0.02)
2787.56/708.89	c INFO: running hybridGM3@0 for 194 with 491 remaining (b = 0.04)
3158.62/802.12	c INFO: running hybridGM3@0 for 194 with 398 remaining (b = 0.03)
3166.59/804.11	c INFO: running TNM@0 for 97 with 396 remaining (b = 0.01)
3565.35/904.42	c INFO: running hybridGM3@388 for 97 with 296 remaining (b = 0.01)
3566.15/904.68	c INFO: running hybridGM3@100 for 97 with 295 remaining (b = 0.01)
3941.00/998.92	c INFO: running hybridGM3@99 for 97 with 201 remaining (b = 0.01)
3952.15/1001.72	c INFO: running TNM@0 for 97 with 198 remaining (b = 0.01)
3968.78/1005.99	c INFO: running hybridGM3@296 for 97 with 194 remaining (b = 0.01)
4318.79/1094.19	c INFO: running hybridGM3@388 for 106 with 106 remaining (b = 0.01)
4346.95/1101.27	c INFO: running hybridGM3@389 for 99 with 99 remaining (b = 0.01)
4350.14/1102.03	c INFO: running hybridGM3@395 for 98 with 98 remaining (b = 0.01)
4729.11/1197.58	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378263-1305464356/watcher-3378263-1305464356 -o /tmp/evaluation-result-3378263-1305464356/solver-3378263-1305464356 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378263-1305464356.cnf -seed 829673185 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.09 8.15 7.26 5/383 7929
/proc/meminfo: memFree=16764916/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=15380 CPUtime=0 cores=1,3,5,7
/proc/7929/stat : 7929 (python) S 7927 7929 7291 0 -1 4202496 818 0 0 0 0 0 0 0 25 0 1 0 658035355 15749120 703 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473965568 0 2147483391 16781312 2 18446744071699215550 0 0 17 3 0 0 0
/proc/7929/statm: 3845 703 334 364 0 332 0

[startup+0.0790339 s]
/proc/loadavg: 8.09 8.15 7.26 5/383 7929
/proc/meminfo: memFree=16764916/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=50008 CPUtime=0.04 cores=1,3,5,7
/proc/7929/stat : 7929 (python) R 7927 7929 7291 0 -1 4202496 1944 0 0 0 3 1 0 0 25 0 1 0 658035355 51208192 1709 33554432000 4194304 5683644 140734881448608 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7929/statm: 12502 1710 476 364 0 1202 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 50008

[startup+0.10103 s]
/proc/loadavg: 8.09 8.15 7.26 5/383 7929
/proc/meminfo: memFree=16764916/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=89764 CPUtime=0.06 cores=1,3,5,7
/proc/7929/stat : 7929 (python) R 7927 7929 7291 0 -1 4202496 2639 0 0 0 5 1 0 0 25 0 1 0 658035355 91918336 2355 33554432000 4194304 5683644 140734881448608 18446744073709551615 274468994071 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7929/statm: 22441 2358 721 364 0 1621 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 89764

[startup+0.301078 s]
/proc/loadavg: 8.09 8.15 7.26 5/383 7929
/proc/meminfo: memFree=16764916/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=135744 CPUtime=0.19 cores=1,3,5,7
/proc/7929/stat : 7929 (python) R 7927 7929 7291 0 -1 4202496 5963 934 1 0 13 6 0 0 18 0 1 0 658035355 139001856 4150 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473965568 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7929/statm: 33936 4150 1032 364 0 3122 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135744

[startup+0.701 s]
/proc/loadavg: 8.09 8.15 7.26 5/383 7929
/proc/meminfo: memFree=16764916/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=283884 CPUtime=0.43 cores=1,3,5,7
/proc/7929/stat : 7929 (python) R 7927 7929 7291 0 -1 4202496 10611 934 1 0 31 12 0 0 15 0 1 0 658035355 290697216 8291 33554432000 4194304 5683644 140734881448608 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7929/statm: 70971 8291 1761 364 0 6895 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 283884

[startup+1.50078 s]
/proc/loadavg: 8.09 8.15 7.26 6/384 7934
/proc/meminfo: memFree=16717540/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=327996 CPUtime=1.23 cores=1,3,5,7
/proc/7929/stat : 7929 (python) R 7927 7929 7291 0 -1 4202496 27455 934 1 0 109 14 0 0 16 0 1 0 658035355 335867904 19384 33554432000 4194304 5683644 140734881448608 18446744073709551615 47174658723277 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7929/statm: 81999 19384 1826 364 0 17923 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 327996

[startup+3.1005 s]
/proc/loadavg: 8.09 8.15 7.26 6/384 7934
/proc/meminfo: memFree=16691872/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=327996 CPUtime=2.83 cores=1,3,5,7
/proc/7929/stat : 7929 (python) R 7927 7929 7291 0 -1 4202496 27455 934 1 0 269 14 0 0 18 0 1 0 658035355 335867904 19384 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473628241 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7929/statm: 81999 19384 1826 364 0 17923 0
Current children cumulated CPU time (s) 2.83
Current children cumulated vsize (KiB) 327996

[startup+6.30094 s]
/proc/loadavg: 7.84 8.10 7.25 6/384 7934
/proc/meminfo: memFree=16692040/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=327996 CPUtime=6.03 cores=1,3,5,7
/proc/7929/stat : 7929 (python) R 7927 7929 7291 0 -1 4202496 27456 934 1 0 589 14 0 0 25 0 1 0 658035355 335867904 19385 33554432000 4194304 5683644 140734881448608 18446744073709551615 4500482 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7929/statm: 81999 19385 1827 364 0 17923 0
Current children cumulated CPU time (s) 6.03
Current children cumulated vsize (KiB) 327996

[startup+12.7008 s]
/proc/loadavg: 7.61 8.05 7.24 9/396 7946
/proc/meminfo: memFree=16618804/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=365128 CPUtime=11.16 cores=1,3,5,7
/proc/7929/stat : 7929 (python) S 7927 7929 7291 0 -1 4202496 30196 934 1 0 1100 16 0 0 20 0 5 0 658035355 373891072 18471 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/7929/statm: 91282 18471 1873 364 0 27191 0
[pid=7935] ppid=7929 vsize=324116 CPUtime=0 cores=1,3,5,7
/proc/7935/stat : 7935 (python) S 7929 7929 7291 0 -1 4202560 731 0 0 0 0 0 0 0 18 0 1 0 658036494 331894784 16935 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987599 0 0 16781312 514 4261691405514047487 0 0 17 7 0 0 0
/proc/7935/statm: 81029 16935 353 364 0 16947 0
[pid=7937] ppid=7935 vsize=276240 CPUtime=1.28 cores=1,3,5,7
/proc/7937/stat : 7937 (hybridGM3) R 7935 7937 7937 0 -1 4202496 4745 0 0 0 128 0 0 0 25 0 1 0 658036495 282869760 4481 33554432000 134512640 135204385 4291615056 18446744073709551615 134543968 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7937/statm: 69060 4481 57 169 0 68890 0
[pid=7938] ppid=7929 vsize=334372 CPUtime=0 cores=1,3,5,7
/proc/7938/stat : 7938 (python) S 7929 7929 7291 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 658036496 342396928 16933 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 4261695253804744703 0 0 17 7 0 0 0
/proc/7938/statm: 83593 16933 347 364 0 19508 0
[pid=7940] ppid=7938 vsize=276232 CPUtime=1.27 cores=1,3,5,7
/proc/7940/stat : 7940 (hybridGM3) R 7938 7940 7940 0 -1 4202496 4733 0 0 0 127 0 0 0 25 0 1 0 658036496 282861568 4470 33554432000 134512640 135204385 4287179808 18446744073709551615 134543968 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7940/statm: 69058 4470 57 169 0 68888 0
[pid=7941] ppid=7929 vsize=344628 CPUtime=0 cores=1,3,5,7
/proc/7941/stat : 7941 (python) S 7929 7929 7291 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 658036497 352899072 16937 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 10578130345511616511 0 0 17 7 0 0 0
/proc/7941/statm: 86157 16937 347 364 0 22069 0
[pid=7943] ppid=7941 vsize=268416 CPUtime=1.26 cores=1,3,5,7
/proc/7943/stat : 7943 (hybridGM3) R 7941 7943 7943 0 -1 4202496 4707 0 0 0 125 1 0 0 25 0 1 0 658036497 274857984 4447 33554432000 134512640 135204385 4292729744 18446744073709551615 134543968 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7943/statm: 67104 4447 57 169 0 66934 0
[pid=7944] ppid=7929 vsize=354884 CPUtime=0 cores=1,3,5,7
/proc/7944/stat : 7944 (python) S 7929 7929 7291 0 -1 4202560 751 0 0 0 0 0 0 0 18 0 1 0 658036498 363401216 16941 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 10578141890383708159 0 0 17 3 0 0 0
/proc/7944/statm: 88721 16941 347 364 0 24630 0
[pid=7946] ppid=7944 vsize=268420 CPUtime=1.25 cores=1,3,5,7
/proc/7946/stat : 7946 (hybridGM3) R 7944 7946 7946 0 -1 4202496 4714 0 0 0 124 1 0 0 25 0 1 0 658036498 274862080 4454 33554432000 134512640 135204385 4290727344 18446744073709551615 134543968 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7946/statm: 67105 4454 57 169 0 66935 0
[pid=7929/tid=7936] ppid=7927 vsize=365128 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7936/stat : 7936 (python) S 7927 7929 7291 0 -1 4202560 27 934 0 0 0 0 0 0 25 0 5 0 658036495 373891072 18471 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 7 0 0 0
[pid=7929/tid=7939] ppid=7927 vsize=365128 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7939/stat : 7939 (python) S 7927 7929 7291 0 -1 4202560 26 934 0 0 0 0 0 0 25 0 5 0 658036496 373891072 18471 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 7 0 0 0
[pid=7929/tid=7942] ppid=7927 vsize=365128 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7942/stat : 7942 (python) S 7927 7929 7291 0 -1 4202560 23 934 0 0 0 0 0 0 25 0 5 0 658036497 373891072 18471 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 7 0 0 0
[pid=7929/tid=7945] ppid=7927 vsize=365128 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7945/stat : 7945 (python) S 7927 7929 7291 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 5 0 658036498 373891072 18471 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 16.22
Current children cumulated vsize (KiB) 2812436

[startup+25.5016 s]
/proc/loadavg: 7.70 8.04 7.25 9/396 7946
/proc/meminfo: memFree=16532288/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=365128 CPUtime=11.16 cores=1,3,5,7
/proc/7929/stat : 7929 (python) S 7927 7929 7291 0 -1 4202496 30196 934 1 0 1100 16 0 0 20 0 5 0 658035355 373891072 18471 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/7929/statm: 91282 18471 1873 364 0 27191 0
[pid=7935] ppid=7929 vsize=324116 CPUtime=0.02 cores=1,3,5,7
/proc/7935/stat : 7935 (python) S 7929 7929 7291 0 -1 4202560 904 0 0 0 1 1 0 0 15 0 1 0 658036494 331894784 16958 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987599 0 0 16781312 514 12806354976334905606 0 0 17 7 0 0 0
/proc/7935/statm: 81029 16958 376 364 0 16947 0
[pid=7937] ppid=7935 vsize=276244 CPUtime=14.09 cores=1,3,5,7
/proc/7937/stat : 7937 (hybridGM3) R 7935 7937 7937 0 -1 4202496 9516 0 0 0 1408 1 0 0 25 0 1 0 658036495 282873856 9252 33554432000 134512640 135204385 4291615056 18446744073709551615 134547775 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7937/statm: 69061 9252 64 169 0 68891 0
[pid=7938] ppid=7929 vsize=334372 CPUtime=0.02 cores=1,3,5,7
/proc/7938/stat : 7938 (python) S 7929 7929 7291 0 -1 4202560 915 0 0 0 1 1 0 0 15 0 1 0 658036496 342396928 16956 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 12806354976334905606 0 0 17 7 0 0 0
/proc/7938/statm: 83593 16956 370 364 0 19508 0
[pid=7940] ppid=7938 vsize=276236 CPUtime=14.05 cores=1,3,5,7
/proc/7940/stat : 7940 (hybridGM3) R 7938 7940 7940 0 -1 4202496 9513 0 0 0 1404 1 0 0 25 0 1 0 658036496 282865664 9250 33554432000 134512640 135204385 4287179808 18446744073709551615 134545859 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7940/statm: 69059 9250 64 169 0 68889 0
[pid=7941] ppid=7929 vsize=344628 CPUtime=0.03 cores=1,3,5,7
/proc/7941/stat : 7941 (python) S 7929 7929 7291 0 -1 4202560 910 0 0 0 2 1 0 0 15 0 1 0 658036497 352899072 16960 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 12806354976334905606 0 0 17 7 0 0 0
/proc/7941/statm: 86157 16960 370 364 0 22069 0
[pid=7943] ppid=7941 vsize=276236 CPUtime=13.78 cores=1,3,5,7
/proc/7943/stat : 7943 (hybridGM3) R 7941 7943 7943 0 -1 4202496 9509 0 0 0 1377 1 0 0 25 0 1 0 658036497 282865664 9249 33554432000 134512640 135204385 4292729744 18446744073709551615 134536646 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7943/statm: 69059 9249 64 169 0 68889 0
[pid=7944] ppid=7929 vsize=354884 CPUtime=0.02 cores=1,3,5,7
/proc/7944/stat : 7944 (python) S 7929 7929 7291 0 -1 4202560 930 0 0 0 1 1 0 0 15 0 1 0 658036498 363401216 16964 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 12806354976334905606 0 0 17 3 0 0 0
/proc/7944/statm: 88721 16964 370 364 0 24630 0
[pid=7946] ppid=7944 vsize=276240 CPUtime=14.06 cores=1,3,5,7
/proc/7946/stat : 7946 (hybridGM3) R 7944 7946 7946 0 -1 4202496 9517 0 0 0 1404 2 0 0 25 0 1 0 658036498 282869760 9257 33554432000 134512640 135204385 4290727344 18446744073709551615 134546082 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7946/statm: 69060 9257 64 169 0 68890 0
[pid=7929/tid=7936] ppid=7927 vsize=365128 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7936/stat : 7936 (python) S 7927 7929 7291 0 -1 4202560 27 934 0 0 0 0 0 0 25 0 5 0 658036495 373891072 18471 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 7 0 0 0
[pid=7929/tid=7939] ppid=7927 vsize=365128 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7939/stat : 7939 (python) S 7927 7929 7291 0 -1 4202560 26 934 0 0 0 0 0 0 25 0 5 0 658036496 373891072 18471 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 7 0 0 0
[pid=7929/tid=7942] ppid=7927 vsize=365128 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7942/stat : 7942 (python) S 7927 7929 7291 0 -1 4202560 23 934 0 0 0 0 0 0 25 0 5 0 658036497 373891072 18471 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 7 0 0 0
[pid=7929/tid=7945] ppid=7927 vsize=365128 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7945/stat : 7945 (python) S 7927 7929 7291 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 5 0 658036498 373891072 18471 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 67.23
Current children cumulated vsize (KiB) 2828084

[startup+51.1011 s]
/proc/loadavg: 7.87 8.06 7.27 9/399 7950
/proc/meminfo: memFree=16501516/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=365128 CPUtime=11.16 cores=1,3,5,7
/proc/7929/stat : 7929 (python) S 7927 7929 7291 0 -1 4202496 30196 934 1 0 1100 16 0 0 20 0 5 0 658035355 373891072 18471 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/7929/statm: 91282 18471 1873 364 0 27191 0
[pid=7935] ppid=7929 vsize=324116 CPUtime=0.1 cores=1,3,5,7
/proc/7935/stat : 7935 (python) S 7929 7929 7291 0 -1 4202560 915 0 0 0 4 6 0 0 15 0 1 0 658036494 331894784 16958 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987599 0 0 16781312 514 12806354976334905606 0 0 17 7 0 0 0
/proc/7935/statm: 81029 16958 376 364 0 16947 0
[pid=7937] ppid=7935 vsize=276244 CPUtime=39.69 cores=1,3,5,7
/proc/7937/stat : 7937 (hybridGM3) R 7935 7937 7937 0 -1 4202496 9516 0 0 0 3968 1 0 0 25 0 1 0 658036495 282873856 9252 33554432000 134512640 135204385 4291615056 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7937/statm: 69061 9252 64 169 0 68891 0
[pid=7938] ppid=7929 vsize=334372 CPUtime=0.09 cores=1,3,5,7
/proc/7938/stat : 7938 (python) S 7929 7929 7291 0 -1 4202560 926 0 0 0 5 4 0 0 15 0 1 0 658036496 342396928 16956 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 12806354976334905606 0 0 17 7 0 0 0
/proc/7938/statm: 83593 16956 370 364 0 19508 0
[pid=7940] ppid=7938 vsize=276236 CPUtime=39.58 cores=1,3,5,7
/proc/7940/stat : 7940 (hybridGM3) R 7938 7940 7940 0 -1 4202496 9513 0 0 0 3957 1 0 0 25 0 1 0 658036496 282865664 9250 33554432000 134512640 135204385 4287179808 18446744073709551615 134546082 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7940/statm: 69059 9250 64 169 0 68889 0
[pid=7941] ppid=7929 vsize=344628 CPUtime=0.1 cores=1,3,5,7
/proc/7941/stat : 7941 (python) S 7929 7929 7291 0 -1 4202560 920 0 0 0 5 5 0 0 15 0 1 0 658036497 352899072 16960 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 12806354976334905606 0 0 17 7 0 0 0
/proc/7941/statm: 86157 16960 370 364 0 22069 0
[pid=7943] ppid=7941 vsize=276236 CPUtime=38.85 cores=1,3,5,7
/proc/7943/stat : 7943 (hybridGM3) R 7941 7943 7943 0 -1 4202496 9511 0 0 0 3884 1 0 0 25 0 1 0 658036497 282865664 9251 33554432000 134512640 135204385 4292729744 18446744073709551615 134536737 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7943/statm: 69059 9251 64 169 0 68889 0
[pid=7944] ppid=7929 vsize=354884 CPUtime=0.09 cores=1,3,5,7
/proc/7944/stat : 7944 (python) S 7929 7929 7291 0 -1 4202560 937 0 0 0 5 4 0 0 15 0 1 0 658036498 363401216 16964 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 12806354976334905606 0 0 17 3 0 0 0
/proc/7944/statm: 88721 16964 370 364 0 24630 0

################
# More data... #
################

Current children cumulated CPU time (s) 4727.52
Current children cumulated vsize (KiB) 11544780

[startup+1197.5 s]
/proc/loadavg: 8.29 8.20 7.91 10/435 8334
/proc/meminfo: memFree=15955252/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=477944 CPUtime=11.51 cores=1,3,5,7
/proc/7929/stat : 7929 (python) S 7927 7929 7291 0 -1 4202496 41429 934 1 0 1130 21 0 0 18 0 16 0 658035355 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/7929/statm: 119486 18546 1904 364 0 55362 0
[pid=7935] ppid=7929 vsize=334360 CPUtime=0.87 cores=1,3,5,7
/proc/7935/stat : 7935 (python) S 7929 7929 7291 0 -1 4202560 1221 0 0 0 43 44 0 0 15 0 2 0 658036494 342384640 17003 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/7935/statm: 83590 17003 416 364 0 19508 0
[pid=7937] ppid=7935 vsize=276244 CPUtime=294.58 cores=1,3,5,7
/proc/7937/stat : 7937 (hybridGM3) T 7935 7937 7937 0 -1 4202496 9518 0 0 0 29457 1 0 0 25 0 1 0 658036495 282873856 9254 33554432000 134512640 135204385 4291615056 18446744073709551615 134547676 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/7937/statm: 69061 9254 64 169 0 68891 0
[pid=7935/tid=7962] ppid=7929 vsize=334360 CPUtime=0 cores=1,3,5,7
/proc/7935/task/7962/stat : 7962 (python) S 7929 7929 7291 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 658046435 342384640 17003 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604466789847488 0 0 -1 7 0 0 0
[pid=7938] ppid=7929 vsize=334372 CPUtime=1.47 cores=1,3,5,7
/proc/7938/stat : 7938 (python) R 7929 7929 7291 0 -1 4202560 1143 0 0 0 70 77 0 0 15 0 2 0 658036496 342396928 16986 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473963444 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/7938/statm: 83593 16986 399 364 0 19508 0
[pid=7940] ppid=7938 vsize=276236 CPUtime=496.21 cores=1,3,5,7
/proc/7940/stat : 7940 (hybridGM3) R 7938 7940 7940 0 -1 4202496 9525 0 0 0 49620 1 0 0 25 0 1 0 658036496 282865664 9262 33554432000 134512640 135204385 4287179808 18446744073709551615 134537357 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7940/statm: 69059 9262 64 169 0 68889 0
[pid=7938/tid=7963] ppid=7929 vsize=334372 CPUtime=0 cores=1,3,5,7
/proc/7938/task/7963/stat : 7963 (python) S 7929 7929 7291 0 -1 4202560 45 0 0 0 0 0 0 0 15 0 2 0 658046436 342396928 16986 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604466789848384 0 0 -1 3 0 0 0
[pid=7941] ppid=7929 vsize=344628 CPUtime=0.88 cores=1,3,5,7
/proc/7941/stat : 7941 (python) S 7929 7929 7291 0 -1 4202560 1139 0 0 0 44 44 0 0 15 0 2 0 658036497 352899072 16990 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/7941/statm: 86157 16990 399 364 0 22069 0
[pid=7943] ppid=7941 vsize=276236 CPUtime=292.55 cores=1,3,5,7
/proc/7943/stat : 7943 (hybridGM3) T 7941 7943 7943 0 -1 4202496 9512 0 0 0 29254 1 0 0 25 0 1 0 658036497 282865664 9252 33554432000 134512640 135204385 4292729744 18446744073709551615 134536646 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/7943/statm: 69059 9252 64 169 0 68889 0
[pid=7941/tid=7964] ppid=7929 vsize=344628 CPUtime=0 cores=1,3,5,7
/proc/7941/task/7964/stat : 7964 (python) S 7929 7929 7291 0 -1 4202560 45 0 0 0 0 0 0 0 15 0 2 0 658046441 352899072 16990 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604468260508032 0 0 -1 7 0 0 0
[pid=7944] ppid=7929 vsize=354884 CPUtime=0.84 cores=1,3,5,7
/proc/7944/stat : 7944 (python) S 7929 7929 7291 0 -1 4202560 1164 0 0 0 43 41 0 0 16 0 2 0 658036498 363401216 16994 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/7944/statm: 88721 16994 399 364 0 24630 0
[pid=7946] ppid=7944 vsize=276240 CPUtime=293.59 cores=1,3,5,7
/proc/7946/stat : 7946 (hybridGM3) T 7944 7946 7946 0 -1 4202496 9517 0 0 0 29357 2 0 0 25 0 1 0 658036498 282869760 9257 33554432000 134512640 135204385 4290727344 18446744073709551615 134536663 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/7946/statm: 69060 9257 64 169 0 68890 0
[pid=7944/tid=7973] ppid=7929 vsize=354884 CPUtime=0 cores=1,3,5,7
/proc/7944/task/7973/stat : 7973 (python) S 7929 7929 7291 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 658056064 363401216 16994 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604468260510720 0 0 -1 1 0 0 0
[pid=7974] ppid=7929 vsize=365140 CPUtime=1.77 cores=1,3,5,7
/proc/7974/stat : 7974 (python) S 7929 7929 7291 0 -1 4202560 1152 0 0 0 96 81 0 0 18 0 2 0 658056065 373903360 16998 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/7974/statm: 91285 16998 399 364 0 27191 0
[pid=7976] ppid=7974 vsize=471116 CPUtime=587.25 cores=1,3,5,7
/proc/7976/stat : 7976 (TNM) R 7974 7976 7976 0 -1 4202496 8432 0 0 0 58721 4 0 0 25 0 1 0 658056066 482422784 8171 33554432000 134512640 135008748 4293576768 18446744073709551615 134527526 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7976/statm: 117779 8171 46 122 0 116679 0
[pid=7974/tid=7985] ppid=7929 vsize=365140 CPUtime=0 cores=1,3,5,7
/proc/7974/task/7985/stat : 7985 (python) S 7929 7929 7291 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658066093 373903360 16998 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604462371619904 0 0 -1 3 0 0 0
[pid=7986] ppid=7929 vsize=375396 CPUtime=1.2 cores=1,3,5,7
/proc/7986/stat : 7986 (python) S 7929 7929 7291 0 -1 4202560 1164 0 0 0 58 62 0 0 16 0 2 0 658066094 384405504 17002 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/7986/statm: 93849 17002 399 364 0 29752 0
[pid=7988] ppid=7986 vsize=276240 CPUtime=393.13 cores=1,3,5,7
/proc/7988/stat : 7988 (hybridGM3) T 7986 7988 7988 0 -1 4202496 9515 0 0 0 39311 2 0 0 25 0 1 0 658066095 282869760 9254 33554432000 134512640 135204385 4291763920 18446744073709551615 134537292 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/7988/statm: 69060 9254 64 169 0 68890 0
[pid=7986/tid=8003] ppid=7929 vsize=375396 CPUtime=0 cores=1,3,5,7
/proc/7986/task/8003/stat : 8003 (python) S 7929 7929 7291 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 658076144 384405504 17002 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604465291396224 0 0 -1 3 0 0 0
[pid=7989] ppid=7929 vsize=385652 CPUtime=1.51 cores=1,3,5,7
/proc/7989/stat : 7989 (python) S 7929 7929 7291 0 -1 4202560 1166 0 0 0 75 76 0 0 15 0 2 0 658066202 394907648 17006 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/7989/statm: 96413 17006 399 364 0 32313 0
[pid=7991] ppid=7989 vsize=276236 CPUtime=483.96 cores=1,3,5,7
/proc/7991/stat : 7991 (hybridGM3) R 7989 7991 7991 0 -1 4202496 9515 0 0 0 48395 1 0 0 25 0 1 0 658066203 282865664 9254 33554432000 134512640 135204385 4286809264 18446744073709551615 134537357 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7991/statm: 69059 9254 64 169 0 68889 0
[pid=7989/tid=8015] ppid=7929 vsize=385652 CPUtime=0 cores=1,3,5,7
/proc/7989/task/8015/stat : 8015 (python) S 7929 7929 7291 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 658086478 394907648 17006 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604465291398016 0 0 -1 3 0 0 0
[pid=7992] ppid=7929 vsize=395908 CPUtime=1.46 cores=1,3,5,7
/proc/7992/stat : 7992 (python) S 7929 7929 7291 0 -1 4202560 1197 0 0 0 69 77 0 0 15 0 2 0 658066404 405409792 17016 33554432000 4194304 5683644 140734881448608 18446744073709551615 274473987654 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/7992/statm: 98977 17016 405 364 0 34874 0
[pid=7994] ppid=7992 vsize=276240 CPUtime=486.41 cores=1,3,5,7
/proc/7994/stat : 7994 (hybridGM3) R 7992 7994 7994 0 -1 4202496 9520 0 0 0 48639 2 0 0 25 0 1 0 658066405 282869760 9254 33554432000 134512640 135204385 4287692512 18446744073709551615 134537292 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7994/statm: 69060 9254 64 169 0 68890 0
[pid=7992/tid=8014] ppid=7929 vsize=395908 CPUtime=0 cores=1,3,5,7
/proc/7992/task/8014/stat : 8014 (python) S 7929 7929 7291 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 658086079 405409792 17016 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604470243937088 0 0 -1 3 0 0 0
[pid=8000] ppid=7929 vsize=406164 CPUtime=0.91 cores=1,3,5,7
/proc/8000/stat : 8000 (python) S 7929 7929 7291 0 -1 4202560 1200 0 0 0 45 46 0 0 16 0 2 0 658075721 415911936 17020 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8000/statm: 101541 17020 405 364 0 37435 0
[pid=8002] ppid=8000 vsize=276240 CPUtime=295.05 cores=1,3,5,7
/proc/8002/stat : 8002 (hybridGM3) T 8000 8002 8002 0 -1 4202496 9521 0 0 0 29503 2 0 0 25 0 1 0 658075722 282869760 9256 33554432000 134512640 135204385 4291995792 18446744073709551615 134537290 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/8002/statm: 69060 9256 64 169 0 68890 0
[pid=8000/tid=8019] ppid=7929 vsize=406164 CPUtime=0 cores=1,3,5,7
/proc/8000/task/8019/stat : 8019 (python) S 7929 7929 7291 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 658095606 415911936 17020 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604466789849280 0 0 -1 1 0 0 0
[pid=8245] ppid=7929 vsize=416420 CPUtime=0.28 cores=1,3,5,7
/proc/8245/stat : 8245 (python) S 7929 7929 7291 0 -1 4202560 1124 0 0 0 15 13 0 0 19 0 2 0 658105533 426414080 17018 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/8245/statm: 104105 17018 399 364 0 39996 0
[pid=8247] ppid=8245 vsize=471116 CPUtime=100.29 cores=1,3,5,7
/proc/8247/stat : 8247 (TNM) T 8245 8247 8247 0 -1 4202496 8436 0 0 0 10027 2 0 0 25 0 1 0 658105533 482422784 8172 33554432000 134512640 135008748 4292254480 18446744073709551615 134529272 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/8247/statm: 117779 8172 46 122 0 116679 0
[pid=8245/tid=8302] ppid=7929 vsize=416420 CPUtime=0 cores=1,3,5,7
/proc/8245/task/8302/stat : 8302 (python) S 7929 7929 7291 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 658115566 426414080 17018 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604459768302272 0 0 -1 3 0 0 0
[pid=8248] ppid=7929 vsize=426676 CPUtime=0.28 cores=1,3,5,7
/proc/8248/stat : 8248 (python) S 7929 7929 7291 0 -1 4202560 1120 0 0 0 15 13 0 0 19 0 2 0 658105737 436916224 17022 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/8248/statm: 106669 17022 399 364 0 42557 0
[pid=8250] ppid=8248 vsize=471116 CPUtime=99.21 cores=1,3,5,7
/proc/8250/stat : 8250 (TNM) T 8248 8250 8250 0 -1 4202496 8440 0 0 0 9919 2 0 0 25 0 1 0 658105738 482422784 8173 33554432000 134512640 135008748 4293411584 18446744073709551615 134529448 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/8250/statm: 117779 8173 46 122 0 116679 0
[pid=8248/tid=8306] ppid=7929 vsize=426676 CPUtime=0 cores=1,3,5,7
/proc/8248/task/8306/stat : 8306 (python) S 7929 7929 7291 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658115766 436916224 17022 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604459768301376 0 0 -1 3 0 0 0
[pid=8251] ppid=7929 vsize=436932 CPUtime=1.17 cores=1,3,5,7
/proc/8251/stat : 8251 (python) S 7929 7929 7291 0 -1 4202560 1196 0 0 0 60 57 0 0 15 0 2 0 658106244 447418368 17026 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/8251/statm: 109233 17026 399 364 0 45118 0
[pid=8253] ppid=8251 vsize=276240 CPUtime=388.85 cores=1,3,5,7
/proc/8253/stat : 8253 (hybridGM3) T 8251 8253 8253 0 -1 4202496 9529 0 0 0 38883 2 0 0 25 0 1 0 658106245 282869760 9262 33554432000 134512640 135204385 4287743488 18446744073709551615 134537403 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8253/statm: 69060 9262 64 169 0 68890 0
[pid=8251/tid=8322] ppid=7929 vsize=436932 CPUtime=0 cores=1,3,5,7
/proc/8251/task/8322/stat : 8322 (python) S 7929 7929 7291 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 658125823 447418368 17026 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604458174697664 0 0 -1 3 0 0 0
[pid=8303] ppid=7929 vsize=447188 CPUtime=0.85 cores=1,3,5,7
/proc/8303/stat : 8303 (python) S 7929 7929 7291 0 -1 4202560 1195 0 0 0 46 39 0 0 15 0 2 0 658115567 457920512 17030 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/8303/statm: 111797 17030 399 364 0 47679 0
[pid=8305] ppid=8303 vsize=276240 CPUtime=291.97 cores=1,3,5,7
/proc/8305/stat : 8305 (hybridGM3) T 8303 8305 8305 0 -1 4202496 9526 0 0 0 29195 2 0 0 25 0 1 0 658115568 282869760 9253 33554432000 134512640 135204385 4290035568 18446744073709551615 134536737 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/8305/statm: 69060 9253 64 169 0 68890 0
[pid=8303/tid=8325] ppid=7929 vsize=447188 CPUtime=0 cores=1,3,5,7
/proc/8303/task/8325/stat : 8325 (python) S 7929 7929 7291 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 658135246 457920512 17030 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604458174700352 0 0 -1 3 0 0 0
[pid=8307] ppid=7929 vsize=457444 CPUtime=0.29 cores=1,3,5,7
/proc/8307/stat : 8307 (python) S 7929 7929 7291 0 -1 4202560 1151 0 0 0 15 14 0 0 19 0 2 0 658115767 468422656 17034 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/8307/statm: 114361 17034 399 364 0 50240 0
[pid=8309] ppid=8307 vsize=471116 CPUtime=100.28 cores=1,3,5,7
/proc/8309/stat : 8309 (TNM) T 8307 8309 8309 0 -1 4202496 8442 0 0 0 10027 1 0 0 25 0 1 0 658115767 482422784 8171 33554432000 134512640 135008748 4294041408 18446744073709551615 134527475 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/8309/statm: 117779 8171 46 122 0 116679 0
[pid=8307/tid=8321] ppid=7929 vsize=457444 CPUtime=0 cores=1,3,5,7
/proc/8307/task/8321/stat : 8321 (python) S 7929 7929 7291 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 658125796 468422656 17034 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604466673669376 0 0 -1 5 0 0 0
[pid=8326] ppid=7929 vsize=467700 CPUtime=0.3 cores=1,3,5,7
/proc/8326/stat : 8326 (python) S 7929 7929 7291 0 -1 4202560 1211 0 0 0 16 14 0 0 19 0 2 0 658135527 478924800 17046 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/8326/statm: 116925 17046 407 364 0 52801 0
[pid=8328] ppid=8326 vsize=471116 CPUtime=100.19 cores=1,3,5,7
/proc/8328/stat : 8328 (TNM) T 8326 8328 8328 0 -1 4202496 8450 0 0 0 10018 1 0 0 25 0 1 0 658135528 482422784 8173 33554432000 134512640 135008748 4291687936 18446744073709551615 134529430 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8328/statm: 117779 8173 46 122 0 116679 0
[pid=8326/tid=8332] ppid=7929 vsize=467700 CPUtime=0 cores=1,3,5,7
/proc/8326/task/8332/stat : 8332 (python) S 7929 7929 7291 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 658145557 478924800 17046 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 514 18446604470904381760 0 0 -1 3 0 0 0
[pid=7929/tid=7936] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7936/stat : 7936 (python) S 7927 7929 7291 0 -1 4202560 37 934 0 0 0 0 0 0 18 0 16 0 658036495 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 7 0 0 0
[pid=7929/tid=7939] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7939/stat : 7939 (python) S 7927 7929 7291 0 -1 4202560 40 934 0 0 0 0 0 0 18 0 16 0 658036496 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 7 0 0 0
[pid=7929/tid=7942] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7942/stat : 7942 (python) S 7927 7929 7291 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 16 0 658036497 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 7 0 0 0
[pid=7929/tid=7945] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7945/stat : 7945 (python) S 7927 7929 7291 0 -1 4202560 33 934 0 0 0 0 0 0 18 0 16 0 658036498 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
[pid=7929/tid=7975] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7975/stat : 7975 (python) S 7927 7929 7291 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 16 0 658056066 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
[pid=7929/tid=7987] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7987/stat : 7987 (python) S 7927 7929 7291 0 -1 4202560 36 934 0 0 0 0 0 0 18 0 16 0 658066095 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
[pid=7929/tid=7990] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7990/stat : 7990 (python) S 7927 7929 7291 0 -1 4202560 36 934 0 0 0 0 0 0 18 0 16 0 658066203 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
[pid=7929/tid=7993] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/7993/stat : 7993 (python) S 7927 7929 7291 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 16 0 658066404 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
[pid=7929/tid=8001] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/8001/stat : 8001 (python) S 7927 7929 7291 0 -1 4202560 22 934 0 0 0 0 0 0 18 0 16 0 658075722 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
[pid=7929/tid=8246] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/8246/stat : 8246 (python) S 7927 7929 7291 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 16 0 658105533 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
[pid=7929/tid=8249] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/8249/stat : 8249 (python) S 7927 7929 7291 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 16 0 658105738 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
[pid=7929/tid=8252] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/8252/stat : 8252 (python) S 7927 7929 7291 0 -1 4202560 35 934 0 0 0 0 0 0 18 0 16 0 658106245 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
[pid=7929/tid=8304] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/8304/stat : 8304 (python) S 7927 7929 7291 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 16 0 658115567 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
[pid=7929/tid=8308] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/8308/stat : 8308 (python) S 7927 7929 7291 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 16 0 658115767 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
[pid=7929/tid=8327] ppid=7927 vsize=477944 CPUtime=0 cores=1,3,5,7
/proc/7929/task/8327/stat : 8327 (python) S 7927 7929 7291 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 16 0 658135527 489414656 18546 33554432000 4194304 5683644 140734881448608 18446744073709551615 274485791153 0 0 16781312 2 18446604459768300480 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4729.11
Current children cumulated vsize (KiB) 11544780

[startup+1197.6 s]
/proc/loadavg: 8.29 8.20 7.91 10/435 8334
/proc/meminfo: memFree=15955252/32950928 swapFree=67111348/67111528
[pid=7929] ppid=7927 vsize=354824 CPUtime=4729.72 cores=1,3,5,7
/proc/7929/stat : 7929 (python) R 7927 7929 7291 0 -1 4202496 50185 156342 1 0 1133 22 471072 745 17 0 1 0 658035355 363339776 18468 33554432000 4194304 5683644 140734881448608 18446744073709551615 4501813 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7929/statm: 88706 18468 1874 364 0 24630 0
Current children cumulated CPU time (s) 4729.72
Current children cumulated vsize (KiB) 354824

Child status: 0
Real time (s): 1197.61
CPU time (s): 4729.73
CPU user time (s): 4722.06
CPU system time (s): 7.67783
CPU usage (%): 394.931
Max. virtual memory (cumulated for all children) (KiB): 11544784

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4722.06
system time used= 7.67783
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 206819
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9315
involuntary context switches= 39109

runsolver used 8.05177 second user time and 18.1372 second system time

The end

Launcher Data 

Begin job on node125 at 2011-05-15 14:59:17
IDJOB=3378263
IDBENCH=82672
IDSOLVER=1859
FILE ID=node125/3378263-1305464356
RUNJOBID= node125-1305462615-7310
PBS_JOBID= 13324969
Free space on /tmp= 73152 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v50000-c210000-S517749870-003-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378263-1305464356/watcher-3378263-1305464356 -o /tmp/evaluation-result-3378263-1305464356/solver-3378263-1305464356 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378263-1305464356.cnf -seed 829673185 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4033245e3bd21ace8cfcd57089ff73c5
RANDOM SEED=829673185

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      16765320 kB
Buffers:        292968 kB
Cached:        4761448 kB
SwapCached:          0 kB
Active:       12096192 kB
Inactive:      3695476 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      16765320 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:           36164 kB
Writeback:           0 kB
AnonPages:    10736812 kB
Mapped:          40900 kB
Slab:           281136 kB
PageTables:      47996 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 81530896 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73124 MiB
End job on node125 at 2011-05-15 15:19:15