Trace number 3321827

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 7807.86 1103.44

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v50000-c210000-S758437099-099-UNKNOWN.cnf
MD5SUM94d10ca78c6cee5b0badfb96b777ac08
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark838.323
Satisfiable
(Un)Satisfiability was proved
Number of variables49999
Number of clauses210000
Sum of the clauses size630000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3210000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.33/0.68	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.33/0.68	c INFO: solving HOME/instance-3321827-1304367998.cnf
0.33/0.68	c INFO: computing task features
1.10/1.40	c INFO: computed variable-clause graph statistics
1.10/1.43	c INFO: computed clause balance statistics
4.80/5.16	c INFO: computed variable balance statistics
7.80/8.12	c INFO: computed small-clause counts
7.80/8.14	c INFO: computed Horn-clause counts
10.90/11.21	c INFO: computed variable graph statistics
10.90/11.21	c INFO: collected features for HOME/instance-3321827-1304367998.cnf in 10.51 s
10.90/11.25	c INFO: running hybridGM3@0 for 97 with 39989 remaining (b = 0.33)
10.90/11.29	c INFO: running hybridGM3@0 for 97 with 39989 remaining (b = 0.21)
11.00/11.33	c INFO: running hybridGM3@0 for 97 with 39989 remaining (b = 0.14)
11.00/11.37	c INFO: running hybridGM3@0 for 194 with 39989 remaining (b = 0.17)
11.10/11.41	c INFO: running hybridGM3@0 for 194 with 39989 remaining (b = 0.13)
11.10/11.45	c INFO: running hybridGM3@0 for 194 with 39989 remaining (b = 0.10)
11.10/11.49	c INFO: running hybridGM3@0 for 194 with 39989 remaining (b = 0.07)
11.20/11.53	c INFO: running TNM@0 for 97 with 39989 remaining (b = 0.03)
711.56/111.62	c INFO: running hybridGM3@0 for 97 with 39889 remaining (b = 0.03)
711.56/111.66	c INFO: running hybridGM3@0 for 97 with 39889 remaining (b = 0.03)
755.56/117.96	c INFO: running hybridGM3@0 for 194 with 39882 remaining (b = 0.05)
879.84/135.74	c INFO: running hybridGM3@0 for 97 with 39864 remaining (b = 0.02)
1376.34/206.87	c INFO: running hybridGM3@97 for 97 with 39793 remaining (b = 0.02)
1384.03/207.95	c INFO: running hybridGM3@0 for 97 with 39792 remaining (b = 0.02)
1418.20/212.84	c INFO: running hybridGM3@100 for 97 with 39787 remaining (b = 0.02)
1468.48/220.04	c INFO: running hybridGM3@0 for 97 with 39780 remaining (b = 0.02)
1557.85/232.81	c INFO: running hybridGM3@0 for 97 with 39767 remaining (b = 0.02)
1795.93/267.00	c INFO: running hybridGM3@94 for 97 with 39733 remaining (b = 0.02)
2083.01/308.08	c INFO: running hybridGM3@293 for 97 with 39692 remaining (b = 0.02)
2090.68/309.16	c INFO: running hybridGM3@98 for 97 with 39691 remaining (b = 0.02)
2133.21/315.24	c INFO: running hybridGM3@0 for 194 with 39685 remaining (b = 0.04)
2138.78/316.05	c INFO: running hybridGM3@0 for 97 with 39684 remaining (b = 0.02)
2248.39/331.75	c INFO: running hybridGM3@98 for 97 with 39668 remaining (b = 0.02)
2264.46/334.03	c INFO: running hybridGM3@96 for 97 with 39666 remaining (b = 0.02)
2510.18/368.14	c INFO: running hybridGM3@298 for 97 with 39632 remaining (b = 0.02)
2772.75/403.28	c INFO: running hybridGM3@393 for 97 with 39597 remaining (b = 0.01)
2780.20/404.27	c INFO: running TNM@100 for 1940 with 39596 remaining (b = 0.23)
2780.91/404.37	c INFO: running TNM@0 for 97 with 39596 remaining (b = 0.02)
2913.20/423.28	c INFO: running hybridGM3@193 for 97 with 39577 remaining (b = 0.02)
2955.20/429.24	c INFO: running hybridGM3@190 for 97 with 39571 remaining (b = 0.02)
3107.15/451.00	c INFO: running hybridGM3@0 for 194 with 39549 remaining (b = 0.03)
3412.40/493.41	c INFO: running hybridGM3@0 for 194 with 39507 remaining (b = 0.03)
3489.89/504.51	c INFO: running TNM@0 for 97 with 39496 remaining (b = 0.01)
3525.47/509.66	c INFO: running hybridGM3@392 for 97 with 39491 remaining (b = 0.01)
3543.05/511.90	c INFO: running hybridGM3@0 for 194 with 39488 remaining (b = 0.02)
3724.29/536.48	c INFO: running TNM@0 for 97 with 39464 remaining (b = 0.01)
3738.93/538.55	c INFO: running hybridGM3@0 for 194 with 39462 remaining (b = 0.02)
4200.47/604.77	c INFO: running adaptg2wsat2009++@0 for 194 with 39395 remaining (b = 0.02)
4305.78/618.92	c INFO: running TNM@0 for 97 with 39381 remaining (b = 0.01)
4438.73/636.76	c INFO: running TNM@0 for 97 with 39363 remaining (b = 0.01)
4773.40/682.83	c INFO: running hybridGM3@100 for 97 with 39317 remaining (b = 0.01)
4830.91/690.55	c INFO: running hybridGM3@100 for 97 with 39310 remaining (b = 0.01)
5126.66/732.62	c INFO: running hybridGM3@99 for 97 with 39268 remaining (b = 0.01)
5326.05/761.27	c INFO: running hybridGM3@0 for 194 with 39239 remaining (b = 0.01)
5349.02/764.54	c INFO: running hybridGM3@99 for 97 with 39236 remaining (b = 0.01)
5366.39/767.09	c INFO: running TNM@0 for 97 with 39233 remaining (b = 0.01)
5527.42/790.11	c INFO: running hybridGM3@295 for 194 with 39210 remaining (b = 0.02)
5539.26/791.82	c INFO: running hybridGM3@298 for 194 with 39208 remaining (b = 0.02)
5790.26/827.87	c INFO: running TNM@0 for 97 with 39172 remaining (b = 0.01)
5939.97/849.32	c INFO: running TNM@0 for 97 with 39151 remaining (b = 0.01)
6063.06/865.82	c INFO: running adaptg2wsat2009++@95 for 97 with 39134 remaining (b = 0.01)
6296.68/897.43	c INFO: running adaptg2wsat2009++@0 for 194 with 39103 remaining (b = 0.01)
6729.10/955.63	c INFO: running TNM@0 for 97 with 39045 remaining (b = 0.01)
6748.52/958.23	c INFO: running TNM@0 for 97 with 39042 remaining (b = 0.01)
6827.91/969.19	c INFO: running TNM@0 for 97 with 39031 remaining (b = 0.01)
6959.38/987.17	c INFO: running TNM@0 for 97 with 39013 remaining (b = 0.01)
6962.18/987.60	c INFO: running hybridGM3@396 for 97 with 39013 remaining (b = 0.01)
7065.31/1001.31	c INFO: running hybridGM3@393 for 97 with 38999 remaining (b = 0.01)
7517.64/1061.96	c INFO: running TNM@0 for 97 with 38938 remaining (b = 0.01)
7581.19/1070.67	c INFO: running TNM@0 for 97 with 38930 remaining (b = 0.01)
7615.24/1075.53	c INFO: running TNM@0 for 97 with 38925 remaining (b = 0.01)
7750.07/1094.93	c INFO: running adaptg2wsat2009++@0 for 194 with 38905 remaining (b = 0.01)
7803.58/1102.64	c INFO: running TNM@0 for 97 with 38898 remaining (b = 0.01)
7807.86/1103.39	c INFO: running hybridGM3@195 for 291 with 38897 remaining (b = 0.02)
7807.86/1103.39	Traceback (most recent call last):
7807.86/1103.39	  File "HOME/solve", line 33, in <module>
7807.86/1103.39	    plac.call(main)
7807.86/1103.39	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321827-1304367998/watcher-3321827-1304367998 -o /tmp/evaluation-result-3321827-1304367998/solver-3321827-1304367998 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321827-1304367998.cnf -seed 548430009 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.49 8.93 7.54 3/237 7913
/proc/meminfo: memFree=18648576/32950928 swapFree=67111528/67111528
[pid=7913] ppid=7911 vsize=15384 CPUtime=0 cores=0-7
/proc/7913/stat : 7913 (python) R 7911 7913 7242 0 -1 4202496 800 0 0 0 0 0 0 0 24 0 1 0 4409243 15753216 686 33554432000 4194304 5683644 140735986038320 18446744073709551615 4349408 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/7913/statm: 3846 692 331 364 0 333 0

[startup+0.072956 s]
/proc/loadavg: 8.49 8.93 7.54 3/237 7913
/proc/meminfo: memFree=18648576/32950928 swapFree=67111528/67111528
[pid=7913] ppid=7911 vsize=36840 CPUtime=0.03 cores=0-7
/proc/7913/stat : 7913 (python) S 7911 7913 7242 0 -1 4202496 1622 0 0 0 3 0 0 0 24 0 1 0 4409243 37724160 1452 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880563200 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/7913/statm: 9210 1452 452 364 0 1027 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 36840

[startup+0.100958 s]
/proc/loadavg: 8.49 8.93 7.54 3/237 7913
/proc/meminfo: memFree=18648576/32950928 swapFree=67111528/67111528
[pid=7913] ppid=7911 vsize=87708 CPUtime=0.05 cores=0-7
/proc/7913/stat : 7913 (python) R 7911 7913 7242 0 -1 4202496 2638 0 0 0 4 1 0 0 17 0 1 0 4409243 89812992 2354 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880563200 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/7913/statm: 21927 2354 717 364 0 1623 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 87708

[startup+0.300911 s]
/proc/loadavg: 8.49 8.93 7.54 3/237 7913
/proc/meminfo: memFree=18648576/32950928 swapFree=67111528/67111528
[pid=7913] ppid=7911 vsize=135752 CPUtime=0.18 cores=0-7
/proc/7913/stat : 7913 (python) R 7911 7913 7242 0 -1 4202496 5964 934 1 0 13 5 0 0 17 0 1 0 4409243 139010048 4151 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880561157 0 2147483391 16781312 2 0 0 0 17 6 0 0 0
/proc/7913/statm: 33938 4151 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 135752

[startup+0.700844 s]
/proc/loadavg: 8.49 8.93 7.54 3/237 7913
/proc/meminfo: memFree=18648576/32950928 swapFree=67111528/67111528
[pid=7913] ppid=7911 vsize=283888 CPUtime=0.4 cores=0-7
/proc/7913/stat : 7913 (python) R 7911 7913 7242 0 -1 4202496 10603 934 1 0 30 10 0 0 16 0 1 0 4409243 290701312 8283 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880251752 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/7913/statm: 70972 8284 1761 364 0 6896 0
Current children cumulated CPU time (s) 0.4
Current children cumulated vsize (KiB) 283888

[startup+1.50069 s]
/proc/loadavg: 8.49 8.93 7.54 3/238 7918
/proc/meminfo: memFree=18602316/32950928 swapFree=67111528/67111528
[pid=7913] ppid=7911 vsize=327996 CPUtime=1.2 cores=0-7
/proc/7913/stat : 7913 (python) R 7911 7913 7242 0 -1 4202496 27454 934 1 0 107 13 0 0 17 0 1 0 4409243 335867904 19383 33554432000 4194304 5683644 140735986038320 18446744073709551615 47121589173540 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/7913/statm: 81999 19383 1826 364 0 17923 0
Current children cumulated CPU time (s) 1.2
Current children cumulated vsize (KiB) 327996

[startup+3.10043 s]
/proc/loadavg: 8.49 8.93 7.54 3/238 7918
/proc/meminfo: memFree=18577144/32950928 swapFree=67111528/67111528
[pid=7913] ppid=7911 vsize=327996 CPUtime=2.8 cores=0-7
/proc/7913/stat : 7913 (python) R 7911 7913 7242 0 -1 4202496 27454 934 1 0 267 13 0 0 19 0 1 0 4409243 335867904 19383 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880223970 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/7913/statm: 81999 19383 1826 364 0 17923 0
Current children cumulated CPU time (s) 2.8
Current children cumulated vsize (KiB) 327996

[startup+6.30084 s]
/proc/loadavg: 7.97 8.82 7.51 3/238 7918
/proc/meminfo: memFree=18577304/32950928 swapFree=67111528/67111528
[pid=7913] ppid=7911 vsize=327996 CPUtime=6 cores=0-7
/proc/7913/stat : 7913 (python) R 7911 7913 7242 0 -1 4202496 27455 934 1 0 587 13 0 0 25 0 1 0 4409243 335867904 19384 33554432000 4194304 5683644 140735986038320 18446744073709551615 4570176 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/7913/statm: 81999 19384 1827 364 0 17923 0
Current children cumulated CPU time (s) 6
Current children cumulated vsize (KiB) 327996

[startup+12.7016 s]
/proc/loadavg: 7.50 8.70 7.48 10/262 7942
/proc/meminfo: memFree=18431188/32950928 swapFree=67111528/67111528
[pid=7913] ppid=7911 vsize=406152 CPUtime=11.24 cores=0-7
/proc/7913/stat : 7913 (python) S 7911 7913 7242 0 -1 4202496 34272 934 1 0 1108 16 0 0 21 0 9 0 4409243 415899648 18502 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892391883 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/7913/statm: 101538 18502 1889 364 0 37435 0
[pid=7919] ppid=7913 vsize=324116 CPUtime=0 cores=0-7
/proc/7919/stat : 7919 (python) S 7913 7913 7242 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 4410369 331894784 16934 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585231 0 0 16781312 514 4219829699309731839 0 0 17 0 0 0 0
/proc/7919/statm: 81029 16934 353 364 0 16947 0
[pid=7921] ppid=7919 vsize=276236 CPUtime=1.43 cores=0-7
/proc/7921/stat : 7921 (hybridGM3) R 7919 7921 7921 0 -1 4202496 4825 0 0 0 142 1 0 0 25 0 1 0 4410369 282865664 4557 33554432000 134512640 135204385 4287092320 18446744073709551615 134543968 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7921/statm: 69059 4557 57 169 0 68889 0
[pid=7922] ppid=7913 vsize=334372 CPUtime=0 cores=0-7
/proc/7922/stat : 7922 (python) S 7913 7913 7242 0 -1 4202560 740 0 0 0 0 0 0 0 18 0 1 0 4410373 342396928 16932 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 1899420085173354495 0 0 17 0 0 0 0
/proc/7922/statm: 83593 16932 347 364 0 19508 0
[pid=7924] ppid=7922 vsize=276236 CPUtime=1.39 cores=0-7
/proc/7924/stat : 7924 (hybridGM3) R 7922 7924 7924 0 -1 4202496 4784 0 0 0 138 1 0 0 25 0 1 0 4410373 282865664 4521 33554432000 134512640 135204385 4294184496 18446744073709551615 134543699 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7924/statm: 69059 4521 57 169 0 68889 0
[pid=7925] ppid=7913 vsize=344628 CPUtime=0 cores=0-7
/proc/7925/stat : 7925 (python) S 7913 7913 7242 0 -1 4202560 744 0 0 0 0 0 0 0 18 0 1 0 4410377 352899072 16936 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 4219825851019034623 0 0 17 0 0 0 0
/proc/7925/statm: 86157 16936 347 364 0 22069 0
[pid=7927] ppid=7925 vsize=276232 CPUtime=1.35 cores=0-7
/proc/7927/stat : 7927 (hybridGM3) R 7925 7927 7927 0 -1 4202496 4786 0 0 0 135 0 0 0 25 0 1 0 4410377 282861568 4523 33554432000 134512640 135204385 4293566736 18446744073709551615 134543975 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/7927/statm: 69058 4523 57 169 0 68888 0
[pid=7928] ppid=7913 vsize=354884 CPUtime=0 cores=0-7
/proc/7928/stat : 7928 (python) S 7913 7913 7242 0 -1 4202560 757 0 0 0 0 0 0 0 18 0 1 0 4410381 363401216 16940 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 10382321893133778943 0 0 17 4 0 0 0
/proc/7928/statm: 88721 16940 347 364 0 24630 0
[pid=7930] ppid=7928 vsize=276232 CPUtime=1.31 cores=0-7
/proc/7930/stat : 7930 (hybridGM3) R 7928 7930 7930 0 -1 4202496 4755 0 0 0 131 0 0 0 25 0 1 0 4410381 282861568 4492 33554432000 134512640 135204385 4288974064 18446744073709551615 134543968 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/7930/statm: 69058 4492 57 169 0 68888 0
[pid=7931] ppid=7913 vsize=365140 CPUtime=0 cores=0-7
/proc/7931/stat : 7931 (python) S 7913 7913 7242 0 -1 4202560 764 0 0 0 0 0 0 0 18 0 1 0 4410385 373903360 16944 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 4097960380242853887 0 0 17 6 0 0 0
/proc/7931/statm: 91285 16944 347 364 0 27191 0
[pid=7933] ppid=7931 vsize=268416 CPUtime=0.81 cores=0-7
/proc/7933/stat : 7933 (hybridGM3) R 7931 7933 7933 0 -1 4202496 4392 0 0 0 81 0 0 0 23 0 1 0 4410385 274857984 4129 33554432000 134512640 135204385 4287851616 18446744073709551615 134543975 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7933/statm: 67104 4129 57 169 0 66934 0
[pid=7934] ppid=7913 vsize=375396 CPUtime=0 cores=0-7
/proc/7934/stat : 7934 (python) S 7913 7913 7242 0 -1 4202560 762 0 0 0 0 0 0 0 18 0 1 0 4410389 384405504 16948 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 5861869226410639359 0 0 17 4 0 0 0
/proc/7934/statm: 93849 16948 347 364 0 29752 0
[pid=7936] ppid=7934 vsize=268420 CPUtime=1.22 cores=0-7
/proc/7936/stat : 7936 (hybridGM3) R 7934 7936 7936 0 -1 4202496 4700 0 0 0 122 0 0 0 25 0 1 0 4410389 274862080 4434 33554432000 134512640 135204385 4290419344 18446744073709551615 134543968 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/7936/statm: 67105 4434 57 169 0 66935 0
[pid=7937] ppid=7913 vsize=385652 CPUtime=0 cores=0-7
/proc/7937/stat : 7937 (python) S 7913 7913 7242 0 -1 4202560 773 0 0 0 0 0 0 0 18 0 1 0 4410393 394907648 16952 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 10382314196552384511 0 0 17 6 0 0 0
/proc/7937/statm: 96413 16952 347 364 0 32313 0
[pid=7939] ppid=7937 vsize=268416 CPUtime=1.15 cores=0-7
/proc/7939/stat : 7939 (hybridGM3) R 7937 7939 7939 0 -1 4202496 4648 0 0 0 114 1 0 0 25 0 1 0 4410395 274857984 4384 33554432000 134512640 135204385 4289422592 18446744073709551615 134543968 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/7939/statm: 67104 4384 57 169 0 66934 0
[pid=7940] ppid=7913 vsize=395908 CPUtime=0 cores=0-7
/proc/7940/stat : 7940 (python) S 7913 7913 7242 0 -1 4202560 784 0 0 0 0 0 0 0 20 0 1 0 4410397 405409792 16958 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 8452807629247348735 0 0 17 1 0 0 0
/proc/7940/statm: 98977 16958 349 364 0 34874 0
[pid=7942] ppid=7940 vsize=465836 CPUtime=0.45 cores=0-7
/proc/7942/stat : 7942 (TNM) R 7940 7942 7942 0 -1 4202496 5135 0 0 0 44 1 0 0 25 0 1 0 4410407 477016064 4864 33554432000 134512640 135008748 4291166592 18446744073709551615 134523389 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7942/statm: 116459 4864 45 122 0 115359 0
[pid=7913/tid=7920] ppid=7911 vsize=406152 CPUtime=0 cores=0-7
/proc/7913/task/7920/stat : 7920 (python) S 7911 7913 7242 0 -1 4202560 26 934 0 0 0 0 0 0 25 0 9 0 4410369 415899648 18502 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 0 0 0 0
[pid=7913/tid=7923] ppid=7911 vsize=406152 CPUtime=0 cores=0-7
/proc/7913/task/7923/stat : 7923 (python) S 7911 7913 7242 0 -1 4202560 23 934 0 0 0 0 0 0 25 0 9 0 4410373 415899648 18502 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 4 0 0 0
[pid=7913/tid=7926] ppid=7911 vsize=406152 CPUtime=0 cores=0-7
/proc/7913/task/7926/stat : 7926 (python) S 7911 7913 7242 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 9 0 4410377 415899648 18502 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 4 0 0 0
[pid=7913/tid=7929] ppid=7911 vsize=406152 CPUtime=0 cores=0-7
/proc/7913/task/7929/stat : 7929 (python) S 7911 7913 7242 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 9 0 4410381 415899648 18502 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 4 0 0 0
[pid=7913/tid=7932] ppid=7911 vsize=406152 CPUtime=0 cores=0-7
/proc/7913/task/7932/stat : 7932 (python) S 7911 7913 7242 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 9 0 4410385 415899648 18502 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7935] ppid=7911 vsize=406152 CPUtime=0 cores=0-7
/proc/7913/task/7935/stat : 7935 (python) S 7911 7913 7242 0 -1 4202560 23 934 0 0 0 0 0 0 22 0 9 0 4410389 415899648 18502 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 4 0 0 0
[pid=7913/tid=7938] ppid=7911 vsize=406152 CPUtime=0 cores=0-7
/proc/7913/task/7938/stat : 7938 (python) S 7911 7913 7242 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 9 0 4410393 415899648 18502 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7941] ppid=7911 vsize=406152 CPUtime=0 cores=0-7
/proc/7913/task/7941/stat : 7941 (python) S 7911 7913 7242 0 -1 4202560 31 934 0 0 0 0 0 0 21 0 9 0 4410397 415899648 18502 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 20.35
Current children cumulated vsize (KiB) 5662272

[startup+25.5013 s]
/proc/loadavg: 7.83 8.72 7.50 10/262 7943
/proc/meminfo: memFree=18254388/32950928 swapFree=67111528/67111528
[pid=7913] ppid=7911 vsize=406152 CPUtime=11.24 cores=0-7
/proc/7913/stat : 7913 (python) S 7911 7913 7242 0 -1 4202496 34272 934 1 0 1108 16 0 0 21 0 9 0 4409243 415899648 18502 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892391883 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/7913/statm: 101538 18502 1889 364 0 37435 0
[pid=7919] ppid=7913 vsize=324116 CPUtime=0.01 cores=0-7
/proc/7919/stat : 7919 (python) S 7913 7913 7242 0 -1 4202560 907 0 0 0 1 0 0 0 15 0 1 0 4410369 331894784 16957 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585231 0 0 16781312 514 6235092946606063873 0 0 17 0 0 0 0
/proc/7919/statm: 81029 16957 376 364 0 16947 0
[pid=7921] ppid=7919 vsize=276240 CPUtime=14.23 cores=0-7
/proc/7921/stat : 7921 (hybridGM3) R 7919 7921 7921 0 -1 4202496 9523 0 0 0 1422 1 0 0 25 0 1 0 4410369 282869760 9255 33554432000 134512640 135204385 4287092320 18446744073709551615 134547654 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7921/statm: 69060 9255 64 169 0 68890 0
[pid=7922] ppid=7913 vsize=334372 CPUtime=0.01 cores=0-7
/proc/7922/stat : 7922 (python) S 7913 7913 7242 0 -1 4202560 911 0 0 0 0 1 0 0 15 0 1 0 4410373 342396928 16955 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 6235092946606063874 0 0 17 0 0 0 0
/proc/7922/statm: 83593 16955 370 364 0 19508 0
[pid=7924] ppid=7922 vsize=276240 CPUtime=14.05 cores=0-7
/proc/7924/stat : 7924 (hybridGM3) R 7922 7924 7924 0 -1 4202496 9513 0 0 0 1404 1 0 0 25 0 1 0 4410373 282869760 9250 33554432000 134512640 135204385 4294184496 18446744073709551615 134545851 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7924/statm: 69060 9250 64 169 0 68890 0
[pid=7925] ppid=7913 vsize=344628 CPUtime=0.02 cores=0-7
/proc/7925/stat : 7925 (python) S 7913 7913 7242 0 -1 4202560 928 0 0 0 1 1 0 0 15 0 1 0 4410377 352899072 16959 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 6235092946606063873 0 0 17 0 0 0 0
/proc/7925/statm: 86157 16959 370 364 0 22069 0
[pid=7927] ppid=7925 vsize=276236 CPUtime=14.15 cores=0-7
/proc/7927/stat : 7927 (hybridGM3) R 7925 7927 7927 0 -1 4202496 9517 0 0 0 1414 1 0 0 25 0 1 0 4410377 282865664 9254 33554432000 134512640 135204385 4293566736 18446744073709551615 134537292 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/7927/statm: 69059 9254 64 169 0 68889 0
[pid=7928] ppid=7913 vsize=354884 CPUtime=0.01 cores=0-7
/proc/7928/stat : 7928 (python) S 7913 7913 7242 0 -1 4202560 949 0 0 0 1 0 0 0 15 0 1 0 4410381 363401216 16963 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 6235092946606063876 0 0 17 4 0 0 0
/proc/7928/statm: 88721 16963 370 364 0 24630 0
[pid=7930] ppid=7928 vsize=276236 CPUtime=14.07 cores=0-7
/proc/7930/stat : 7930 (hybridGM3) R 7928 7930 7930 0 -1 4202496 9514 0 0 0 1406 1 0 0 25 0 1 0 4410381 282865664 9251 33554432000 134512640 135204385 4288974064 18446744073709551615 134537540 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/7930/statm: 69059 9251 64 169 0 68889 0
[pid=7931] ppid=7913 vsize=365140 CPUtime=0.02 cores=0-7
/proc/7931/stat : 7931 (python) S 7913 7913 7242 0 -1 4202560 971 0 0 0 1 1 0 0 18 0 1 0 4410385 373903360 16967 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 6235092946606063876 0 0 17 6 0 0 0
/proc/7931/statm: 91285 16967 370 364 0 27191 0
[pid=7933] ppid=7931 vsize=276236 CPUtime=7.21 cores=0-7
/proc/7933/stat : 7933 (hybridGM3) R 7931 7933 7933 0 -1 4202496 9510 0 0 0 720 1 0 0 25 0 1 0 4410385 282865664 9247 33554432000 134512640 135204385 4287851616 18446744073709551615 134537357 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7933/statm: 69059 9247 64 169 0 68889 0

################
# More data... #
################

/proc/8026/stat : 8026 (python) S 7913 7913 7242 0 -1 4202560 1174 0 0 0 14 12 0 0 19 0 2 0 4492031 630050816 17374 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892391883 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/8026/statm: 153821 17374 407 364 0 89673 0
[pid=8028] ppid=8026 vsize=471116 CPUtime=97.94 cores=0-7
/proc/8028/stat : 8028 (TNM) T 8026 8028 8028 0 -1 4202496 8450 0 0 0 9793 1 0 0 25 0 1 0 4492032 482422784 8170 33554432000 134512640 135008748 4290339984 18446744073709551615 134529475 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/8028/statm: 117779 8170 46 122 0 116679 0
[pid=8026/tid=8044] ppid=7913 vsize=615284 CPUtime=0 cores=0-7
/proc/8026/task/8044/stat : 8044 (python) S 7913 7913 7242 0 -1 4202560 96 0 0 0 0 0 0 0 19 0 2 0 4505056 630050816 17374 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 514 18446604453863001152 0 0 -1 0 0 0 0
[pid=8031] ppid=7913 vsize=625540 CPUtime=0.22 cores=0-7
/proc/8031/stat : 8031 (python) S 7913 7913 7242 0 -1 4202560 1182 0 0 0 12 10 0 0 19 0 2 0 4494176 640552960 17379 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892391883 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/8031/statm: 156385 17379 406 364 0 92234 0
[pid=8033] ppid=8031 vsize=471116 CPUtime=102.93 cores=0-7
/proc/8033/stat : 8033 (TNM) T 8031 8033 8033 0 -1 4202496 8455 0 0 0 10291 2 0 0 25 0 1 0 4494176 482422784 8170 33554432000 134512640 135008748 4292474912 18446744073709551615 134527111 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/8033/statm: 117779 8170 46 122 0 116679 0
[pid=8031/tid=8040] ppid=7913 vsize=625540 CPUtime=0 cores=0-7
/proc/8031/task/8040/stat : 8040 (python) S 7913 7913 7242 0 -1 4202560 95 0 0 0 0 0 0 0 19 0 2 0 4504797 640552960 17379 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 514 18446604453885944192 0 0 -1 4 0 0 0
[pid=8036] ppid=7913 vsize=635796 CPUtime=0.43 cores=0-7
/proc/8036/stat : 8036 (python) S 7913 7913 7242 0 -1 4202560 1103 0 0 0 24 19 0 0 18 0 1 0 4498987 651055104 17255 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 6235092946606063873 0 0 17 1 0 0 0
/proc/8036/statm: 158949 17255 371 364 0 94795 0
[pid=8038] ppid=8036 vsize=438604 CPUtime=186.03 cores=0-7
/proc/8038/stat : 8038 (adaptg2wsat2009) R 8036 8038 8038 0 -1 4202496 8789 0 0 0 18601 2 0 0 25 0 1 0 4498988 449130496 8498 33554432000 134512640 135107411 4289737840 18446744073709551615 134531239 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/8038/statm: 109651 8498 59 146 0 108527 0
[pid=8041] ppid=7913 vsize=646052 CPUtime=0.22 cores=0-7
/proc/8041/stat : 8041 (python) S 7913 7913 7242 0 -1 4202560 1241 0 0 0 12 10 0 0 19 0 2 0 4504807 661557248 17289 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892391883 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/8041/statm: 161513 17289 400 364 0 97356 0
[pid=8043] ppid=8041 vsize=471116 CPUtime=101.23 cores=0-7
/proc/8043/stat : 8043 (TNM) T 8041 8043 8043 0 -1 4202496 8458 0 0 0 10122 1 0 0 25 0 1 0 4504807 482422784 8172 33554432000 134512640 135008748 4289267680 18446744073709551615 134527524 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/8043/statm: 117779 8172 46 122 0 116679 0
[pid=8041/tid=8058] ppid=7913 vsize=646052 CPUtime=0 cores=0-7
/proc/8041/task/8058/stat : 8058 (python) S 7913 7913 7242 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 4515429 661557248 17289 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 514 18446604444765000064 0 0 -1 1 0 0 0
[pid=8045] ppid=7913 vsize=656308 CPUtime=0.24 cores=0-7
/proc/8045/stat : 8045 (python) S 7913 7913 7242 0 -1 4202560 1244 0 0 0 11 13 0 0 19 0 2 0 4505066 672059392 17293 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892391883 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/8045/statm: 164077 17293 400 364 0 99917 0
[pid=8047] ppid=8045 vsize=471116 CPUtime=102.2 cores=0-7
/proc/8047/stat : 8047 (TNM) T 8045 8047 8047 0 -1 4202496 8457 0 0 0 10218 2 0 0 25 0 1 0 4505077 482422784 8170 33554432000 134512640 135008748 4293973488 18446744073709551615 134529292 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/8047/statm: 117779 8170 46 122 0 116679 0
[pid=8045/tid=8063] ppid=7913 vsize=656308 CPUtime=0 cores=0-7
/proc/8045/task/8063/stat : 8063 (python) S 7913 7913 7242 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 4516300 672059392 17293 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 514 18446604446290832192 0 0 -1 1 0 0 0
[pid=8049] ppid=7913 vsize=666564 CPUtime=0.23 cores=0-7
/proc/8049/stat : 8049 (python) S 7913 7913 7242 0 -1 4202560 1235 0 0 0 11 12 0 0 19 0 2 0 4506163 682561536 17298 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892391883 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/8049/statm: 166641 17298 401 364 0 102478 0
[pid=8051] ppid=8049 vsize=471116 CPUtime=102.92 cores=0-7
/proc/8051/stat : 8051 (TNM) T 8049 8051 8051 0 -1 4202496 8460 0 0 0 10291 1 0 0 25 0 1 0 4506163 482422784 8172 33554432000 134512640 135008748 4289824112 18446744073709551615 134527444 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8051/statm: 117779 8172 46 122 0 116679 0
[pid=8049/tid=8067] ppid=7913 vsize=666564 CPUtime=0 cores=0-7
/proc/8049/task/8067/stat : 8067 (python) S 7913 7913 7242 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 4516785 682561536 17298 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 514 18446604446290829504 0 0 -1 1 0 0 0
[pid=8053] ppid=7913 vsize=676820 CPUtime=0.24 cores=0-7
/proc/8053/stat : 8053 (python) S 7913 7913 7242 0 -1 4202560 1128 0 0 0 11 13 0 0 18 0 1 0 4507960 693063680 17277 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 6235092946606063874 0 0 17 1 0 0 0
/proc/8053/statm: 169205 17277 377 364 0 105039 0
[pid=8055] ppid=8053 vsize=471116 CPUtime=82.06 cores=0-7
/proc/8055/stat : 8055 (TNM) R 8053 8055 8055 0 -1 4202496 8463 0 0 0 8205 1 0 0 25 0 1 0 4507961 482422784 8172 33554432000 134512640 135008748 4293999744 18446744073709551615 134529448 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/8055/statm: 117779 8172 46 122 0 116679 0
[pid=8059] ppid=7913 vsize=687076 CPUtime=0.08 cores=0-7
/proc/8059/stat : 8059 (python) S 7913 7913 7242 0 -1 4202560 1136 0 0 0 3 5 0 0 18 0 1 0 4515440 703565824 17282 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 6235092946606063873 0 0 17 1 0 0 0
/proc/8059/statm: 171769 17282 378 364 0 107600 0
[pid=8061] ppid=8059 vsize=471116 CPUtime=38.31 cores=0-7
/proc/8061/stat : 8061 (TNM) R 8059 8061 8061 0 -1 4202496 8464 0 0 0 3830 1 0 0 25 0 1 0 4515440 482422784 8172 33554432000 134512640 135008748 4286802704 18446744073709551615 134529367 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/8061/statm: 117779 8172 46 122 0 116679 0
[pid=8064] ppid=7913 vsize=697332 CPUtime=0.06 cores=0-7
/proc/8064/stat : 8064 (python) S 7913 7913 7242 0 -1 4202560 1129 0 0 0 2 4 0 0 18 0 1 0 4516311 714067968 17286 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 6235092946606063873 0 0 17 1 0 0 0
/proc/8064/statm: 174333 17286 378 364 0 110161 0
[pid=8066] ppid=8064 vsize=471116 CPUtime=27.23 cores=0-7
/proc/8066/stat : 8066 (TNM) R 8064 8066 8066 0 -1 4202496 8463 0 0 0 2722 1 0 0 25 0 1 0 4516311 482422784 8170 33554432000 134512640 135008748 4293580640 18446744073709551615 134527526 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/8066/statm: 117779 8170 46 122 0 116679 0
[pid=8068] ppid=7913 vsize=707588 CPUtime=0.05 cores=0-7
/proc/8068/stat : 8068 (python) S 7913 7913 7242 0 -1 4202560 1134 0 0 0 2 3 0 0 18 0 1 0 4516796 724570112 17289 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 6235092946606063873 0 0 17 1 0 0 0
/proc/8068/statm: 176897 17289 377 364 0 112722 0
[pid=8070] ppid=8068 vsize=471116 CPUtime=13.81 cores=0-7
/proc/8070/stat : 8070 (TNM) R 8068 8070 8070 0 -1 4202496 8463 0 0 0 1379 2 0 0 25 0 1 0 4516797 482422784 8171 33554432000 134512640 135008748 4290353392 18446744073709551615 134529367 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/8070/statm: 117779 8171 46 122 0 116679 0
[pid=8071] ppid=7913 vsize=717844 CPUtime=0.01 cores=0-7
/proc/8071/stat : 8071 (python) S 7913 7913 7242 0 -1 4202560 1120 0 0 0 1 0 0 0 18 0 1 0 4518736 735072256 17286 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 1861184293439504643 0 0 17 6 0 0 0
/proc/8071/statm: 179461 17286 370 364 0 115283 0
[pid=8073] ppid=8071 vsize=438604 CPUtime=8.28 cores=0-7
/proc/8073/stat : 8073 (adaptg2wsat2009) R 8071 8073 8073 0 -1 4202496 8793 0 0 0 827 1 0 0 25 0 1 0 4518737 449130496 8497 33554432000 134512640 135107411 4286934736 18446744073709551615 134531709 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/8073/statm: 109651 8497 59 146 0 108527 0
[pid=8074] ppid=7913 vsize=728100 CPUtime=0 cores=0-7
/proc/8074/stat : 8074 (python) S 7913 7913 7242 0 -1 4202560 938 0 0 0 0 0 0 0 19 0 1 0 4519507 745574400 17265 33554432000 4194304 5683644 140735986038320 18446744073709551615 218880585286 0 0 16781312 514 1861191990020931583 0 0 17 6 0 0 0
/proc/8074/statm: 182025 17265 345 364 0 117844 0
[pid=8076] ppid=8074 vsize=465836 CPUtime=0.57 cores=0-7
/proc/8076/stat : 8076 (TNM) R 8074 8076 8076 0 -1 4202496 5156 0 0 0 56 1 0 0 22 0 1 0 4519508 477016064 4865 33554432000 134512640 135008748 4291761680 18446744073709551615 134523307 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/8076/statm: 116459 4865 45 122 0 115359 0
[pid=7913/tid=7920] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7920/stat : 7920 (python) S 7911 7913 7242 0 -1 4202560 42 934 0 0 0 0 0 0 15 0 35 0 4410369 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 0 0 0 0
[pid=7913/tid=7923] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7923/stat : 7923 (python) S 7911 7913 7242 0 -1 4202560 27 934 0 0 0 0 0 0 18 0 35 0 4410373 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 4 0 0 0
[pid=7913/tid=7926] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7926/stat : 7926 (python) S 7911 7913 7242 0 -1 4202560 36 934 0 0 0 0 0 0 18 0 35 0 4410377 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 4 0 0 0
[pid=7913/tid=7929] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7929/stat : 7929 (python) S 7911 7913 7242 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 35 0 4410381 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 4 0 0 0
[pid=7913/tid=7932] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7932/stat : 7932 (python) S 7911 7913 7242 0 -1 4202560 40 934 0 0 0 0 0 0 18 0 35 0 4410385 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7935] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7935/stat : 7935 (python) S 7911 7913 7242 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 35 0 4410389 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 4 0 0 0
[pid=7913/tid=7938] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7938/stat : 7938 (python) S 7911 7913 7242 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 35 0 4410393 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7941] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7941/stat : 7941 (python) S 7911 7913 7242 0 -1 4202560 35 934 0 0 0 0 0 0 18 0 35 0 4410397 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7950] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7950/stat : 7950 (python) S 7911 7913 7242 0 -1 4202560 33 934 0 0 0 0 0 0 18 0 35 0 4421040 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7962] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7962/stat : 7962 (python) S 7911 7913 7242 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 35 0 4440768 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7970] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7970/stat : 7970 (python) S 7911 7913 7242 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 35 0 4449681 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7974] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7974/stat : 7974 (python) S 7911 7913 7242 0 -1 4202560 53 934 0 0 0 0 0 0 18 0 35 0 4454343 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7978] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7978/stat : 7978 (python) S 7911 7913 7242 0 -1 4202560 46 934 0 0 0 0 0 0 18 0 35 0 4458584 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7981] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7981/stat : 7981 (python) S 7911 7913 7242 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 35 0 4459695 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7986] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7986/stat : 7986 (python) S 7911 7913 7242 0 -1 4202560 36 934 0 0 0 0 0 0 18 0 35 0 4460434 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7990] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7990/stat : 7990 (python) S 7911 7913 7242 0 -1 4202560 89 934 0 0 0 0 0 0 20 0 35 0 4462892 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7993] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7993/stat : 7993 (python) S 7911 7913 7242 0 -1 4202560 40 934 0 0 0 0 0 0 18 0 35 0 4463099 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=7999] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/7999/stat : 7999 (python) S 7911 7913 7242 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 35 0 4469721 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8002] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8002/stat : 8002 (python) S 7911 7913 7242 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 35 0 4471136 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8007] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8007/stat : 8007 (python) S 7911 7913 7242 0 -1 4202560 24 934 0 0 0 0 0 0 19 0 35 0 4472920 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 3 0 0 0
[pid=7913/tid=8017] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8017/stat : 8017 (python) S 7911 7913 7242 0 -1 4202560 44 934 0 0 0 0 0 0 19 0 35 0 4485371 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 3 0 0 0
[pid=7913/tid=8022] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8022/stat : 8022 (python) S 7911 7913 7242 0 -1 4202560 39 934 0 0 0 0 0 0 19 0 35 0 4485953 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8027] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8027/stat : 8027 (python) S 7911 7913 7242 0 -1 4202560 39 934 0 0 0 0 0 0 19 0 35 0 4492031 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8032] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8032/stat : 8032 (python) S 7911 7913 7242 0 -1 4202560 34 934 0 0 0 0 0 0 19 0 35 0 4494176 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8037] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8037/stat : 8037 (python) S 7911 7913 7242 0 -1 4202560 28 934 0 0 0 0 0 0 19 0 35 0 4498987 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8042] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8042/stat : 8042 (python) S 7911 7913 7242 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 35 0 4504807 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8046] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8046/stat : 8046 (python) S 7911 7913 7242 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 35 0 4505067 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8050] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8050/stat : 8050 (python) S 7911 7913 7242 0 -1 4202560 26 934 0 0 0 0 0 0 19 0 35 0 4506163 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8054] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8054/stat : 8054 (python) S 7911 7913 7242 0 -1 4202560 29 934 0 0 0 0 0 0 19 0 35 0 4507961 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8060] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8060/stat : 8060 (python) S 7911 7913 7242 0 -1 4202560 26 934 0 0 0 0 0 0 19 0 35 0 4515440 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8065] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8065/stat : 8065 (python) S 7911 7913 7242 0 -1 4202560 23 934 0 0 0 0 0 0 19 0 35 0 4516311 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8069] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8069/stat : 8069 (python) S 7911 7913 7242 0 -1 4202560 24 934 0 0 0 0 0 0 19 0 35 0 4516797 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8072] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8072/stat : 8072 (python) S 7911 7913 7242 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 35 0 4518737 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
[pid=7913/tid=8075] ppid=7911 vsize=738344 CPUtime=0 cores=0-7
/proc/7913/task/8075/stat : 8075 (python) S 7911 7913 7242 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 35 0 4519508 756064256 18866 33554432000 4194304 5683644 140735986038320 18446744073709551615 218892388785 0 0 16781312 2 18446604446740080448 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 7807.86
Current children cumulated vsize (KiB) 31757572

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 7913 and gives
#  childrusage.ru_utime.tv_sec=15
#  childrusage.ru_utime.tv_usec=251681
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=522920
# CPU time returned by wait4() is 15.7746
# while last known CPU time is 7807.86
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1103.44
CPU time (s): 7807.86
CPU user time (s): 7798.83
CPU system time (s): 9.03
CPU usage (%): 707.594
Max. virtual memory (cumulated for all children) (KiB): 31757572

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 15.2517
system time used= 0.52292
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 72366
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3548
involuntary context switches= 66

runsolver used 8.03978 second user time and 19.903 second system time

The end

Launcher Data 

Begin job on node112 at 2011-05-02 22:26:38
IDJOB=3321827
IDBENCH=82605
IDSOLVER=1712
FILE ID=node112/3321827-1304367998
RUNJOBID= node112-1304366468-7256
PBS_JOBID= 13181758
Free space on /tmp= 72396 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v50000-c210000-S758437099-099-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321827-1304367998/watcher-3321827-1304367998 -o /tmp/evaluation-result-3321827-1304367998/solver-3321827-1304367998 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321827-1304367998.cnf -seed 548430009 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 94d10ca78c6cee5b0badfb96b777ac08
RANDOM SEED=548430009

node112.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5180.45
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      18648856 kB
Buffers:         80704 kB
Cached:        1363456 kB
SwapCached:          0 kB
Active:       13043380 kB
Inactive:      1105772 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      18648856 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           35708 kB
Writeback:           0 kB
AnonPages:    12705528 kB
Mapped:          18664 kB
Slab:            55108 kB
PageTables:      34272 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 14365704 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72364 MiB
End job on node112 at 2011-05-02 22:45:04