Trace number 3321780

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 10911.4 1375.11

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v30000-c126000-S913491256-073-UNKNOWN.cnf
MD5SUM785309e098f4d0ba8da3027c8426e931
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark222.412
Satisfiable
(Un)Satisfiability was proved
Number of variables30000
Number of clauses126000
Sum of the clauses size378000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3126000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.31/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.31/0.47	c INFO: solving HOME/instance-3321780-1304367105.cnf
0.31/0.47	c INFO: computing task features
0.80/0.92	c INFO: computed variable-clause graph statistics
0.80/0.94	c INFO: computed clause balance statistics
2.99/3.14	c INFO: computed variable balance statistics
4.69/4.89	c INFO: computed small-clause counts
4.79/4.90	c INFO: computed Horn-clause counts
5.89/6.07	c INFO: computed variable graph statistics
5.89/6.07	c INFO: collected features for HOME/instance-3321780-1304367105.cnf in 5.57 s
5.99/6.11	c INFO: running hybridGM3@0 for 97 with 39994 remaining (b = 0.38)
5.99/6.15	c INFO: running hybridGM3@0 for 97 with 39994 remaining (b = 0.23)
5.99/6.19	c INFO: running hybridGM3@0 for 97 with 39994 remaining (b = 0.15)
6.09/6.22	c INFO: running hybridGM3@0 for 194 with 39994 remaining (b = 0.20)
6.09/6.26	c INFO: running hybridGM3@0 for 194 with 39994 remaining (b = 0.16)
6.19/6.30	c INFO: running hybridGM3@0 for 194 with 39994 remaining (b = 0.12)
6.19/6.34	c INFO: running TNM@0 for 97 with 39994 remaining (b = 0.05)
6.19/6.38	c INFO: running hybridGM3@0 for 194 with 39994 remaining (b = 0.08)
807.20/106.56	c INFO: running hybridGM3@0 for 194 with 39893 remaining (b = 0.06)
811.96/107.14	c INFO: running hybridGM3@0 for 97 with 39893 remaining (b = 0.03)
811.96/107.19	c INFO: running hybridGM3@0 for 97 with 39893 remaining (b = 0.03)
812.71/107.23	c INFO: running hybridGM3@0 for 97 with 39893 remaining (b = 0.03)
1571.99/202.33	c INFO: running hybridGM3@100 for 97 with 39798 remaining (b = 0.03)
1571.99/202.37	c INFO: running hybridGM3@100 for 97 with 39798 remaining (b = 0.03)
1572.79/202.41	c INFO: running hybridGM3@95 for 97 with 39798 remaining (b = 0.03)
1579.92/203.40	c INFO: running hybridGM3@0 for 97 with 39797 remaining (b = 0.03)
1580.75/203.47	c INFO: running hybridGM3@0 for 97 with 39797 remaining (b = 0.03)
1581.52/203.51	c INFO: running hybridGM3@0 for 97 with 39796 remaining (b = 0.03)
1581.52/203.56	c INFO: running hybridGM3@99 for 97 with 39796 remaining (b = 0.03)
2331.86/297.51	c INFO: running hybridGM3@0 for 194 with 39702 remaining (b = 0.05)
2331.86/297.56	c INFO: running hybridGM3@0 for 194 with 39702 remaining (b = 0.04)
2332.57/297.62	c INFO: running hybridGM3@292 for 97 with 39702 remaining (b = 0.02)
2339.82/298.59	c INFO: running TNM@0 for 1940 with 39701 remaining (b = 0.24)
2342.19/298.84	c INFO: running TNM@0 for 97 with 39701 remaining (b = 0.02)
2382.05/303.82	c INFO: running hybridGM3@100 for 97 with 39696 remaining (b = 0.02)
2389.27/304.73	c INFO: running gnovelty+2@0 for 97 with 39695 remaining (b = 0.02)
2389.27/304.79	c INFO: running hybridGM3@0 for 194 with 39695 remaining (b = 0.03)
3132.28/397.86	c INFO: running hybridGM3@386 for 97 with 39602 remaining (b = 0.01)
3141.88/399.06	c INFO: running hybridGM3@0 for 194 with 39601 remaining (b = 0.02)
3189.73/405.04	c INFO: running TNM@0 for 97 with 39595 remaining (b = 0.01)
3190.47/405.11	c INFO: running adaptg2wsat2009++@0 for 97 with 39595 remaining (b = 0.01)
3904.83/494.78	c INFO: running hybridGM3@94 for 97 with 39505 remaining (b = 0.01)
3905.58/494.85	c INFO: running hybridGM3@95 for 97 with 39505 remaining (b = 0.01)
3939.07/499.10	c INFO: running hybridGM3@0 for 194 with 39501 remaining (b = 0.02)
3962.93/502.03	c INFO: running hybridGM3@100 for 97 with 39498 remaining (b = 0.01)
3989.20/505.31	c INFO: running hybridGM3@0 for 194 with 39495 remaining (b = 0.02)
3989.20/505.39	c INFO: running TNM@0 for 97 with 39495 remaining (b = 0.01)
4664.41/590.02	c INFO: running hybridGM3@292 for 97 with 39410 remaining (b = 0.01)
4664.41/590.10	c INFO: running hybridGM3@193 for 97 with 39410 remaining (b = 0.01)
4714.60/596.33	c INFO: running hybridGM3@99 for 97 with 39404 remaining (b = 0.01)
4721.84/597.26	c INFO: running TNM@0 for 97 with 39403 remaining (b = 0.01)
4788.71/605.65	c INFO: running TNM@0 for 97 with 39394 remaining (b = 0.01)
5471.08/691.27	c INFO: running hybridGM3@387 for 194 with 39309 remaining (b = 0.02)
5471.86/691.36	c INFO: running hybridGM3@390 for 97 with 39309 remaining (b = 0.01)
5473.41/691.57	c INFO: running TNM@0 for 97 with 39308 remaining (b = 0.01)
5504.47/695.46	c INFO: running TNM@0 for 97 with 39305 remaining (b = 0.01)
5513.17/696.56	c INFO: running hybridGM3@193 for 97 with 39303 remaining (b = 0.01)
5588.03/705.92	c INFO: running TNM@195 for 1843 with 39294 remaining (b = 0.12)
5593.58/706.69	c INFO: running hybridGM3@99 for 97 with 39293 remaining (b = 0.01)
6230.59/786.61	c INFO: running adaptg2wsat2009++@0 for 194 with 39213 remaining (b = 0.01)
6264.07/790.85	c INFO: running hybridGM3@0 for 194 with 39209 remaining (b = 0.01)
6287.87/793.83	c INFO: running hybridGM3@293 for 194 with 39206 remaining (b = 0.01)
6302.99/795.74	c INFO: running adaptg2wsat2009++@0 for 194 with 39204 remaining (b = 0.01)
6400.06/807.97	c INFO: running hybridGM3@0 for 194 with 39192 remaining (b = 0.01)
6995.65/882.71	c INFO: running FH@0 for 97 with 39117 remaining (b = 0.01)
7794.24/983.08	c INFO: running adaptg2wsat2009++@99 for 97 with 39017 remaining (b = 0.01)
7794.96/983.19	c INFO: running hybridGM3@196 for 291 with 39017 remaining (b = 0.01)
7859.40/991.24	c INFO: running adaptg2wsat2009++@97 for 97 with 39009 remaining (b = 0.01)
7860.13/991.34	c INFO: running hybridGM3@393 for 97 with 39009 remaining (b = 0.01)
7875.28/993.25	c INFO: running hybridGM3@96 for 97 with 39007 remaining (b = 0.01)
7963.66/1004.34	c INFO: running hybridGM3@191 for 97 with 38996 remaining (b = 0.01)
8553.42/1078.42	c INFO: running adaptg2wsat2009++@0 for 194 with 38922 remaining (b = 0.01)
8618.57/1086.63	c INFO: running hybridGM3@196 for 291 with 38913 remaining (b = 0.01)
8657.54/1091.54	c INFO: running adaptg2wsat2009++@293 for 291 with 38909 remaining (b = 0.02)
8713.22/1098.59	c INFO: running hybridGM3@292 for 194 with 38901 remaining (b = 0.01)
8769.66/1105.63	c INFO: running TNM@0 for 97 with 38894 remaining (b = 0.00)
9567.31/1205.95	c INFO: running TNM@0 for 97 with 38794 remaining (b = 0.00)
10140.73/1278.04	c INFO: running TNM@0 for 97 with 38722 remaining (b = 0.00)
10147.04/1278.85	c INFO: running hybridGM3@391 for 97 with 38721 remaining (b = 0.01)
10315.61/1300.10	c INFO: running hybridGM3@393 for 97 with 38700 remaining (b = 0.01)
10364.90/1306.26	c INFO: running hybridGM3@195 for 291 with 38694 remaining (b = 0.01)
10904.90/1374.15	c INFO: running TNM@0 for 97 with 38626 remaining (b = 0.00)
10911.44/1375.07	c INFO: running TNM@0 for 97 with 38625 remaining (b = 0.00)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321780-1304367105/watcher-3321780-1304367105 -o /tmp/evaluation-result-3321780-1304367105/solver-3321780-1304367105 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321780-1304367105.cnf -seed 1155635814 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.60 6.07 3.83 1/214 6834
/proc/meminfo: memFree=18079120/32951124 swapFree=62859076/67111528
[pid=6834] ppid=6832 vsize=15764 CPUtime=0 cores=0-7
/proc/6834/stat : 6834 (python) R 6832 6834 6630 0 -1 4202496 846 0 0 0 0 0 0 0 20 0 1 0 548310967 16142336 728 33554432000 4194304 5683644 140734357152464 18446744073709551615 4972457 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/6834/statm: 3941 728 334 364 0 428 0

[startup+0.096963 s]
/proc/loadavg: 4.60 6.07 3.83 1/214 6834
/proc/meminfo: memFree=18079120/32951124 swapFree=62859076/67111528
[pid=6834] ppid=6832 vsize=101880 CPUtime=0.06 cores=0-7
/proc/6834/stat : 6834 (python) R 6832 6834 6630 0 -1 4202496 2959 0 0 0 5 1 0 0 17 0 1 0 548310967 104325120 2652 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446943232 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/6834/statm: 25470 2652 838 364 0 1770 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 101880

[startup+0.100989 s]
/proc/loadavg: 4.60 6.07 3.83 1/214 6834
/proc/meminfo: memFree=18079120/32951124 swapFree=62859076/67111528
[pid=6834] ppid=6832 vsize=104220 CPUtime=0.06 cores=0-7
/proc/6834/stat : 6834 (python) R 6832 6834 6630 0 -1 4202496 3048 0 0 0 5 1 0 0 17 0 1 0 548310967 106721280 2732 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446943232 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/6834/statm: 26055 2732 847 364 0 1803 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104220

[startup+0.300895 s]
/proc/loadavg: 4.60 6.07 3.83 1/214 6834
/proc/meminfo: memFree=18079120/32951124 swapFree=62859076/67111528
[pid=6834] ppid=6832 vsize=163648 CPUtime=0.23 cores=0-7
/proc/6834/stat : 6834 (python) S 6832 6834 6630 0 -1 4202496 6666 934 1 0 16 7 0 0 17 0 1 0 548310967 167575552 4755 33554432000 4194304 5683644 140734357152464 18446744073709551615 244442021511 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/6834/statm: 40912 4755 1311 364 0 3453 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 163648

[startup+0.700813 s]
/proc/loadavg: 4.60 6.07 3.83 1/214 6834
/proc/meminfo: memFree=18079120/32951124 swapFree=62859076/67111528
[pid=6834] ppid=6832 vsize=294392 CPUtime=0.6 cores=0-7
/proc/6834/stat : 6834 (python) R 6832 6834 6630 0 -1 4202496 13159 934 1 0 49 11 0 0 17 0 1 0 548310967 301457408 10840 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446641641 0 0 16781312 2 0 0 0 17 7 0 0 0
/proc/6834/statm: 73598 10840 1761 364 0 9522 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 294392

[startup+1.5007 s]
/proc/loadavg: 4.60 6.07 3.83 2/215 6839
/proc/meminfo: memFree=18027652/32951124 swapFree=62859076/67111528
[pid=6834] ppid=6832 vsize=307972 CPUtime=1.4 cores=0-7
/proc/6834/stat : 6834 (python) R 6832 6834 6630 0 -1 4202496 20142 934 1 0 128 12 0 0 19 0 1 0 548310967 315363328 14378 33554432000 4194304 5683644 140734357152464 18446744073709551615 47754825266136 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/6834/statm: 76993 14378 1826 364 0 12917 0
Current children cumulated CPU time (s) 1.4
Current children cumulated vsize (KiB) 307972

[startup+3.1004 s]
/proc/loadavg: 4.60 6.07 3.83 2/215 6839
/proc/meminfo: memFree=18027656/32951124 swapFree=62859076/67111528
[pid=6834] ppid=6832 vsize=307972 CPUtime=2.99 cores=0-7
/proc/6834/stat : 6834 (python) R 6832 6834 6630 0 -1 4202496 20142 934 1 0 287 12 0 0 22 0 1 0 548310967 315363328 14378 33554432000 4194304 5683644 140734357152464 18446744073709551615 4623682 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/6834/statm: 76993 14378 1826 364 0 12917 0
Current children cumulated CPU time (s) 2.99
Current children cumulated vsize (KiB) 307972

[startup+6.30077 s]
/proc/loadavg: 4.31 5.98 3.81 2/215 6839
/proc/meminfo: memFree=18028116/32951124 swapFree=62859076/67111528
[pid=6834] ppid=6832 vsize=422452 CPUtime=6.19 cores=0-7
/proc/6834/stat : 6834 (python) R 6832 6834 6630 0 -1 4202496 24348 934 1 0 605 14 0 0 18 0 6 0 548310967 432590848 14011 33554432000 4194304 5683644 140734357152464 18446744073709551615 244450383639 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/6834/statm: 105613 14011 1882 364 0 41519 0
Current children cumulated CPU time (s) 6.19
Current children cumulated vsize (KiB) 422452

[startup+12.7016 s]
/proc/loadavg: 4.60 6.02 3.83 9/239 6863
/proc/meminfo: memFree=17774344/32951124 swapFree=62859076/67111528
[pid=6834] ppid=6832 vsize=453220 CPUtime=6.28 cores=0-7
/proc/6834/stat : 6834 (python) S 6832 6834 6630 0 -1 4202496 26348 934 1 0 613 15 0 0 19 0 9 0 548310967 464097280 14065 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/6834/statm: 113305 14065 1889 364 0 49202 0
[pid=6840] ppid=6834 vsize=305648 CPUtime=0 cores=0-7
/proc/6840/stat : 6840 (python) S 6834 6834 6630 0 -1 4202560 889 0 0 0 0 0 0 0 18 0 1 0 548311578 312983552 12341 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965263 0 0 16781312 514 17300706065943331080 0 0 17 6 0 0 0
/proc/6840/statm: 76412 12341 376 364 0 12330 0
[pid=6842] ppid=6840 vsize=268424 CPUtime=6.57 cores=0-7
/proc/6842/stat : 6842 (hybridGM3) R 6840 6842 6842 0 -1 4202496 7607 0 0 0 657 0 0 0 25 0 1 0 548311579 274866176 7344 33554432000 134512640 135204385 4291976848 18446744073709551615 134536929 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/6842/statm: 67106 7344 64 169 0 66936 0
[pid=6843] ppid=6834 vsize=315904 CPUtime=0 cores=0-7
/proc/6843/stat : 6843 (python) S 6834 6834 6630 0 -1 4202560 902 0 0 0 0 0 0 0 18 0 1 0 548311582 323485696 12339 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 17300709914234028296 0 0 17 1 0 0 0
/proc/6843/statm: 78976 12339 370 364 0 14891 0
[pid=6845] ppid=6843 vsize=268424 CPUtime=6.53 cores=0-7
/proc/6845/stat : 6845 (hybridGM3) R 6843 6845 6845 0 -1 4202496 7606 0 0 0 653 0 0 0 25 0 1 0 548311582 274866176 7344 33554432000 134512640 135204385 4294328912 18446744073709551615 134537403 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/6845/statm: 67106 7344 64 169 0 66936 0
[pid=6846] ppid=6834 vsize=326160 CPUtime=0 cores=0-7
/proc/6846/stat : 6846 (python) S 6834 6834 6630 0 -1 4202560 919 0 0 0 0 0 0 0 18 0 1 0 548311586 333987840 12343 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 4277404526186561800 0 0 17 0 0 0 0
/proc/6846/statm: 81540 12343 370 364 0 17452 0
[pid=6848] ppid=6846 vsize=268420 CPUtime=6.5 cores=0-7
/proc/6848/stat : 6848 (hybridGM3) R 6846 6848 6848 0 -1 4202496 7611 0 0 0 649 1 0 0 25 0 1 0 548311586 274862080 7346 33554432000 134512640 135204385 4293276656 18446744073709551615 134546144 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/6848/statm: 67105 7346 64 169 0 66935 0
[pid=6849] ppid=6834 vsize=336416 CPUtime=0 cores=0-7
/proc/6849/stat : 6849 (python) S 6834 6834 6630 0 -1 4202560 942 0 0 0 0 0 0 0 18 0 1 0 548311590 344489984 12347 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 4414521047873716488 0 0 17 6 0 0 0
/proc/6849/statm: 84104 12347 370 364 0 20013 0
[pid=6851] ppid=6849 vsize=268420 CPUtime=6.46 cores=0-7
/proc/6851/stat : 6851 (hybridGM3) R 6849 6851 6851 0 -1 4202496 7605 0 0 0 645 1 0 0 25 0 1 0 548311590 274862080 7342 33554432000 134512640 135204385 4289109872 18446744073709551615 134537483 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/6851/statm: 67105 7342 64 169 0 66935 0
[pid=6852] ppid=6834 vsize=412208 CPUtime=0 cores=0-7
/proc/6852/stat : 6852 (python) S 6834 6834 6630 0 -1 4202560 799 0 0 0 0 0 0 0 18 0 1 0 548311594 422100992 12525 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 17515348602749157638 0 0 17 0 0 0 0
/proc/6852/statm: 103052 12525 371 364 0 38958 0
[pid=6854] ppid=6852 vsize=268424 CPUtime=6.34 cores=0-7
/proc/6854/stat : 6854 (hybridGM3) R 6852 6854 6854 0 -1 4202496 7601 0 0 0 634 0 0 0 25 0 1 0 548311594 274866176 7346 33554432000 134512640 135204385 4291824368 18446744073709551615 134536744 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/6854/statm: 67106 7346 64 169 0 66936 0
[pid=6855] ppid=6834 vsize=422464 CPUtime=0 cores=0-7
/proc/6855/stat : 6855 (python) S 6834 6834 6630 0 -1 4202560 809 0 0 0 0 0 0 0 18 0 1 0 548311598 432603136 12501 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 17515356299330552072 0 0 17 0 0 0 0
/proc/6855/statm: 105616 12501 371 364 0 41519 0
[pid=6857] ppid=6855 vsize=268424 CPUtime=6.35 cores=0-7
/proc/6857/stat : 6857 (hybridGM3) R 6855 6857 6857 0 -1 4202496 7598 0 0 0 635 0 0 0 25 0 1 0 548311598 274866176 7345 33554432000 134512640 135204385 4290349376 18446744073709551615 134537242 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/6857/statm: 67106 7345 64 169 0 66936 0
[pid=6858] ppid=6834 vsize=432720 CPUtime=0 cores=0-7
/proc/6858/stat : 6858 (python) S 6834 6834 6630 0 -1 4202560 815 0 0 0 0 0 0 0 18 0 1 0 548311602 443105280 12507 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 17515344754458460423 0 0 17 0 0 0 0
/proc/6858/statm: 108180 12507 371 364 0 44080 0
[pid=6860] ppid=6858 vsize=463080 CPUtime=6.25 cores=0-7
/proc/6860/stat : 6860 (TNM) R 6858 6860 6860 0 -1 4202496 5199 0 0 0 625 0 0 0 25 0 1 0 548311604 474193920 4941 33554432000 134512640 135008748 4291538080 18446744073709551615 134524927 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/6860/statm: 115770 4941 46 122 0 114670 0
[pid=6861] ppid=6834 vsize=442976 CPUtime=0 cores=0-7
/proc/6861/stat : 6861 (python) S 6834 6834 6630 0 -1 4202560 863 0 0 0 0 0 0 0 18 0 1 0 548311606 453607424 12549 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 11030723966120460550 0 0 17 0 0 0 0
/proc/6861/statm: 110744 12549 377 364 0 46641 0
[pid=6863] ppid=6861 vsize=268420 CPUtime=6.29 cores=0-7
/proc/6863/stat : 6863 (hybridGM3) R 6861 6863 6863 0 -1 4202496 7604 0 0 0 628 1 0 0 25 0 1 0 548311606 274862080 7342 33554432000 134512640 135204385 4293362912 18446744073709551615 134546073 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/6863/statm: 67105 7342 64 169 0 66935 0
[pid=6834/tid=6841] ppid=6832 vsize=453220 CPUtime=0 cores=0-7
/proc/6834/task/6841/stat : 6841 (python) S 6832 6834 6630 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 9 0 548311578 464097280 14065 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 3 0 0 0
[pid=6834/tid=6844] ppid=6832 vsize=453220 CPUtime=0 cores=0-7
/proc/6834/task/6844/stat : 6844 (python) S 6832 6834 6630 0 -1 4202560 20 934 0 0 0 0 0 0 25 0 9 0 548311582 464097280 14065 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 3 0 0 0
[pid=6834/tid=6847] ppid=6832 vsize=453220 CPUtime=0 cores=0-7
/proc/6834/task/6847/stat : 6847 (python) S 6832 6834 6630 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 9 0 548311586 464097280 14065 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 1 0 0 0
[pid=6834/tid=6850] ppid=6832 vsize=453220 CPUtime=0 cores=0-7
/proc/6834/task/6850/stat : 6850 (python) S 6832 6834 6630 0 -1 4202560 23 934 0 0 0 0 0 0 19 0 9 0 548311590 464097280 14065 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 4 0 0 0
[pid=6834/tid=6853] ppid=6832 vsize=453220 CPUtime=0 cores=0-7
/proc/6834/task/6853/stat : 6853 (python) S 6832 6834 6630 0 -1 4202560 92 934 0 0 0 0 0 0 19 0 9 0 548311594 464097280 14065 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 1 0 0 0
[pid=6834/tid=6856] ppid=6832 vsize=453220 CPUtime=0 cores=0-7
/proc/6834/task/6856/stat : 6856 (python) S 6832 6834 6630 0 -1 4202560 34 934 0 0 0 0 0 0 19 0 9 0 548311598 464097280 14065 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 6 0 0 0
[pid=6834/tid=6859] ppid=6832 vsize=453220 CPUtime=0 cores=0-7
/proc/6834/task/6859/stat : 6859 (python) S 6832 6834 6630 0 -1 4202560 34 934 0 0 0 0 0 0 19 0 9 0 548311602 464097280 14065 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 7 0 0 0
[pid=6834/tid=6862] ppid=6832 vsize=453220 CPUtime=0 cores=0-7
/proc/6834/task/6862/stat : 6862 (python) S 6832 6834 6630 0 -1 4202560 33 934 0 0 0 0 0 0 20 0 9 0 548311606 464097280 14065 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 57.57
Current children cumulated vsize (KiB) 5789752

[startup+25.5012 s]
/proc/loadavg: 5.35 6.11 3.90 9/239 6864
/proc/meminfo: memFree=17776172/32951124 swapFree=62859076/67111528
[pid=6834] ppid=6832 vsize=453220 CPUtime=6.28 cores=0-7
/proc/6834/stat : 6834 (python) S 6832 6834 6630 0 -1 4202496 26348 934 1 0 613 15 0 0 19 0 9 0 548310967 464097280 14065 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/6834/statm: 113305 14065 1889 364 0 49202 0
[pid=6840] ppid=6834 vsize=305648 CPUtime=0.02 cores=0-7
/proc/6840/stat : 6840 (python) S 6834 6834 6630 0 -1 4202560 896 0 0 0 1 1 0 0 18 0 1 0 548311578 312983552 12341 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965263 0 0 16781312 514 14293053026762457352 0 0 17 6 0 0 0
/proc/6840/statm: 76412 12341 376 364 0 12330 0
[pid=6842] ppid=6840 vsize=268424 CPUtime=19.37 cores=0-7
/proc/6842/stat : 6842 (hybridGM3) R 6840 6842 6842 0 -1 4202496 7607 0 0 0 1937 0 0 0 25 0 1 0 548311579 274866176 7344 33554432000 134512640 135204385 4291976848 18446744073709551615 134536744 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/6842/statm: 67106 7344 64 169 0 66936 0
[pid=6843] ppid=6834 vsize=315904 CPUtime=0.02 cores=0-7
/proc/6843/stat : 6843 (python) S 6834 6834 6630 0 -1 4202560 908 0 0 0 1 1 0 0 18 0 1 0 548311582 323485696 12339 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 14293053026762457352 0 0 17 1 0 0 0
/proc/6843/statm: 78976 12339 370 364 0 14891 0
[pid=6845] ppid=6843 vsize=268424 CPUtime=19.33 cores=0-7
/proc/6845/stat : 6845 (hybridGM3) R 6843 6845 6845 0 -1 4202496 7606 0 0 0 1933 0 0 0 25 0 1 0 548311582 274866176 7344 33554432000 134512640 135204385 4294328912 18446744073709551615 134536806 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/6845/statm: 67106 7344 64 169 0 66936 0
[pid=6846] ppid=6834 vsize=326160 CPUtime=0.02 cores=0-7
/proc/6846/stat : 6846 (python) S 6834 6834 6630 0 -1 4202560 923 0 0 0 1 1 0 0 18 0 1 0 548311586 333987840 12343 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 14293053026762457352 0 0 17 0 0 0 0
/proc/6846/statm: 81540 12343 370 364 0 17452 0
[pid=6848] ppid=6846 vsize=268420 CPUtime=19.3 cores=0-7
/proc/6848/stat : 6848 (hybridGM3) R 6846 6848 6848 0 -1 4202496 7613 0 0 0 1929 1 0 0 25 0 1 0 548311586 274862080 7348 33554432000 134512640 135204385 4293276656 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/6848/statm: 67105 7348 64 169 0 66935 0
[pid=6849] ppid=6834 vsize=336416 CPUtime=0.02 cores=0-7
/proc/6849/stat : 6849 (python) S 6834 6834 6630 0 -1 4202560 946 0 0 0 1 1 0 0 18 0 1 0 548311590 344489984 12347 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 14293053026762457352 0 0 17 6 0 0 0
/proc/6849/statm: 84104 12347 370 364 0 20013 0
[pid=6851] ppid=6849 vsize=268420 CPUtime=19.26 cores=0-7
/proc/6851/stat : 6851 (hybridGM3) R 6849 6851 6851 0 -1 4202496 7613 0 0 0 1925 1 0 0 25 0 1 0 548311590 274862080 7350 33554432000 134512640 135204385 4289109872 18446744073709551615 134536744 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/6851/statm: 67105 7350 64 169 0 66935 0
[pid=6852] ppid=6834 vsize=412208 CPUtime=0.02 cores=0-7
/proc/6852/stat : 6852 (python) S 6834 6834 6630 0 -1 4202560 800 0 0 0 1 1 0 0 18 0 1 0 548311594 422100992 12525 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 14293053026762457350 0 0 17 0 0 0 0
/proc/6852/statm: 103052 12525 371 364 0 38958 0
[pid=6854] ppid=6852 vsize=268424 CPUtime=19.03 cores=0-7
/proc/6854/stat : 6854 (hybridGM3) R 6852 6854 6854 0 -1 4202496 7601 0 0 0 1903 0 0 0 25 0 1 0 548311594 274866176 7346 33554432000 134512640 135204385 4291824368 18446744073709551615 134536843 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/6854/statm: 67106 7346 64 169 0 66936 0

################
# More data... #
################

[pid=6942/tid=6958] ppid=6834 vsize=596816 CPUtime=0 cores=0-7
/proc/6942/task/6958/stat : 6958 (python) S 6834 6834 6630 0 -1 4202560 82 0 0 0 0 0 0 0 19 0 2 0 548390043 611139584 12836 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 514 18446604440278876544 0 0 -1 5 0 0 0
[pid=6946] ppid=6834 vsize=607072 CPUtime=0.18 cores=0-7
/proc/6946/stat : 6946 (python) S 6834 6834 6630 0 -1 4202560 1189 0 0 0 9 9 0 0 19 0 2 0 548380513 621641728 12902 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/6946/statm: 151768 12902 407 364 0 87617 0
[pid=6948] ppid=6946 vsize=463080 CPUtime=99.62 cores=0-7
/proc/6948/stat : 6948 (TNM) T 6946 6948 6948 0 -1 4202496 5225 0 0 0 9962 0 0 0 25 0 1 0 548380514 474193920 4941 33554432000 134512640 135008748 4290132672 18446744073709551615 134529367 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/6948/statm: 115770 4941 46 122 0 114670 0
[pid=6946/tid=6962] ppid=6834 vsize=607072 CPUtime=0 cores=0-7
/proc/6946/task/6962/stat : 6962 (python) S 6834 6834 6630 0 -1 4202560 82 0 0 0 0 0 0 0 19 0 2 0 548390531 621641728 12902 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 514 18446604453857728320 0 0 -1 2 0 0 0
[pid=6955] ppid=6834 vsize=617328 CPUtime=0.57 cores=0-7
/proc/6955/stat : 6955 (python) S 6834 6834 6630 0 -1 4202560 1190 0 0 0 29 28 0 0 18 0 2 0 548389628 632143872 12842 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/6955/statm: 154332 12842 401 364 0 90178 0
[pid=6957] ppid=6955 vsize=429460 CPUtime=291.42 cores=0-7
/proc/6957/stat : 6957 (adaptg2wsat2009) T 6955 6957 6957 0 -1 4202496 5427 0 0 0 29142 0 0 0 25 0 1 0 548389629 439767040 5141 33554432000 134512640 135107411 4286704848 18446744073709551615 134531340 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/6957/statm: 107365 5141 59 146 0 106241 0
[pid=6955/tid=6977] ppid=6834 vsize=617328 CPUtime=0 cores=0-7
/proc/6955/task/6977/stat : 6977 (python) S 6834 6834 6630 0 -1 4202560 90 0 0 0 0 0 0 0 18 0 2 0 548409264 632143872 12842 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 514 18446604445964561152 0 0 -1 5 0 0 0
[pid=6959] ppid=6834 vsize=627584 CPUtime=1.17 cores=0-7
/proc/6959/stat : 6959 (python) S 6834 6834 6630 0 -1 4202560 1198 0 0 0 59 58 0 0 18 0 2 0 548390053 642646016 12897 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/6959/statm: 156896 12897 401 364 0 92739 0
[pid=6961] ppid=6959 vsize=268424 CPUtime=557.76 cores=0-7
/proc/6961/stat : 6961 (hybridGM3) R 6959 6961 6961 0 -1 4202496 7637 0 0 0 55774 2 0 0 25 0 1 0 548390053 274866176 7354 33554432000 134512640 135204385 4292105264 18446744073709551615 134537233 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/6961/statm: 67106 7354 64 169 0 66936 0
[pid=6959/tid=6980] ppid=6834 vsize=627584 CPUtime=0 cores=0-7
/proc/6959/task/6980/stat : 6980 (python) S 6834 6834 6630 0 -1 4202560 90 0 0 0 0 0 0 0 18 0 2 0 548410281 642646016 12897 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 514 18446604445964559360 0 0 -1 7 0 0 0
[pid=6963] ppid=6834 vsize=637840 CPUtime=1.15 cores=0-7
/proc/6963/stat : 6963 (python) S 6834 6834 6630 0 -1 4202560 1198 0 0 0 59 56 0 0 18 0 2 0 548390541 653148160 12874 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/6963/statm: 159460 12874 401 364 0 95300 0
[pid=6965] ppid=6963 vsize=429460 CPUtime=578.71 cores=0-7
/proc/6965/stat : 6965 (adaptg2wsat2009) R 6963 6965 6965 0 -1 4202496 5424 0 0 0 57870 1 0 0 25 0 1 0 548390543 439767040 5140 33554432000 134512640 135107411 4287586000 18446744073709551615 134531709 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/6965/statm: 107365 5140 59 146 0 106241 0
[pid=6963/tid=6979] ppid=6834 vsize=637840 CPUtime=0 cores=0-7
/proc/6963/task/6979/stat : 6979 (python) S 6834 6834 6630 0 -1 4202560 90 0 0 0 0 0 0 0 18 0 2 0 548410080 653148160 12874 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 514 18446604453874160896 0 0 -1 7 0 0 0
[pid=6967] ppid=6834 vsize=648096 CPUtime=0.6 cores=0-7
/proc/6967/stat : 6967 (python) S 6834 6834 6630 0 -1 4202560 1211 0 0 0 32 28 0 0 18 0 2 0 548391764 663650304 12965 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/6967/statm: 162024 12965 401 364 0 97861 0
[pid=6969] ppid=6967 vsize=268424 CPUtime=296.87 cores=0-7
/proc/6969/stat : 6969 (hybridGM3) T 6967 6969 6969 0 -1 4202496 7637 0 0 0 29684 3 0 0 25 0 1 0 548391765 274866176 7350 33554432000 134512640 135204385 4289869664 18446744073709551615 134537322 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/6969/statm: 67106 7350 64 169 0 66936 0
[pid=6967/tid=6982] ppid=6834 vsize=648096 CPUtime=0 cores=0-7
/proc/6967/task/6982/stat : 6982 (python) S 6834 6834 6630 0 -1 4202560 88 0 0 0 0 0 0 0 18 0 2 0 548411391 663650304 12965 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 514 18446604453874161792 0 0 -1 2 0 0 0
[pid=6972] ppid=6834 vsize=658352 CPUtime=0.2 cores=0-7
/proc/6972/stat : 6972 (python) S 6834 6834 6630 0 -1 4202560 1229 0 0 0 10 10 0 0 19 0 2 0 548399239 674152448 12896 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/6972/statm: 164588 12896 407 364 0 100422 0
[pid=6974] ppid=6972 vsize=258440 CPUtime=99.22 cores=0-7
/proc/6974/stat : 6974 (FH) T 6972 6974 6974 0 -1 4202496 5359 0 0 0 9919 3 0 0 25 0 1 0 548399239 264642560 5067 33554432000 134512640 134936988 4293930352 18446744073709551615 134545109 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/6974/statm: 64610 5067 41 104 0 64114 0
[pid=6972/tid=6978] ppid=6834 vsize=658352 CPUtime=0 cores=0-7
/proc/6972/task/6978/stat : 6978 (python) S 6834 6834 6630 0 -1 4202560 87 0 0 0 0 0 0 0 19 0 2 0 548409269 674152448 12896 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 514 18446604453868974464 0 0 -1 7 0 0 0
[pid=6985] ppid=6834 vsize=668608 CPUtime=0.41 cores=0-7
/proc/6985/stat : 6985 (python) S 6834 6834 6630 0 -1 4202560 1242 0 0 0 23 18 0 0 19 0 2 0 548418810 684654592 12943 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/6985/statm: 167152 12943 407 364 0 102983 0
[pid=6987] ppid=6985 vsize=429460 CPUtime=199.06 cores=0-7
/proc/6987/stat : 6987 (adaptg2wsat2009) T 6985 6987 6987 0 -1 4202496 5433 0 0 0 19905 1 0 0 25 0 1 0 548418810 439767040 5142 33554432000 134512640 135107411 4294447936 18446744073709551615 134531810 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/6987/statm: 107365 5142 59 146 0 106241 0
[pid=6985/tid=7000] ppid=6834 vsize=668608 CPUtime=0 cores=0-7
/proc/6985/task/7000/stat : 7000 (python) S 6834 6834 6630 0 -1 4202560 92 0 0 0 0 0 0 0 19 0 2 0 548438759 684654592 12943 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 514 18446604453877033984 0 0 -1 0 0 0 0
[pid=6989] ppid=6834 vsize=678864 CPUtime=0.2 cores=0-7
/proc/6989/stat : 6989 (python) S 6834 6834 6630 0 -1 4202560 1259 0 0 0 11 9 0 0 19 0 2 0 548421531 695156736 13018 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/6989/statm: 169716 13018 407 364 0 105544 0
[pid=6991] ppid=6989 vsize=463080 CPUtime=99.77 cores=0-7
/proc/6991/stat : 6991 (TNM) T 6989 6991 6991 0 -1 4202496 5234 0 0 0 9974 3 0 0 25 0 1 0 548421531 474193920 4943 33554432000 134512640 135008748 4291256896 18446744073709551615 134527896 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/6991/statm: 115770 4943 46 122 0 114670 0
[pid=6989/tid=6994] ppid=6834 vsize=678864 CPUtime=0 cores=0-7
/proc/6989/task/6994/stat : 6994 (python) S 6834 6834 6630 0 -1 4202560 93 0 0 0 0 0 0 0 19 0 2 0 548431550 695156736 13018 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 514 18446604453604887232 0 0 -1 1 0 0 0
[pid=6995] ppid=6834 vsize=689120 CPUtime=0.2 cores=0-7
/proc/6995/stat : 6995 (python) S 6834 6834 6630 0 -1 4202560 1258 0 0 0 11 9 0 0 19 0 2 0 548431562 705658880 12991 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/6995/statm: 172280 12991 407 364 0 108105 0
[pid=6997] ppid=6995 vsize=463080 CPUtime=99.84 cores=0-7
/proc/6997/stat : 6997 (TNM) T 6995 6997 6997 0 -1 4202496 5231 0 0 0 9983 1 0 0 25 0 1 0 548431563 474193920 4941 33554432000 134512640 135008748 4287146048 18446744073709551615 134527164 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/6997/statm: 115770 4941 46 122 0 114670 0
[pid=6995/tid=7005] ppid=6834 vsize=689120 CPUtime=0 cores=0-7
/proc/6995/task/7005/stat : 7005 (python) S 6834 6834 6630 0 -1 4202560 89 0 0 0 0 0 0 0 19 0 2 0 548441582 705658880 12991 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 514 18446604449898672512 0 0 -1 2 0 0 0
[pid=7001] ppid=6834 vsize=699376 CPUtime=0.19 cores=0-7
/proc/7001/stat : 7001 (python) S 6834 6834 6630 0 -1 4202560 1038 0 0 0 10 9 0 0 18 0 1 0 548438771 716161024 13055 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 14293053026762457345 0 0 17 0 0 0 0
/proc/7001/statm: 174844 13055 373 364 0 110666 0
[pid=7003] ppid=7001 vsize=463080 CPUtime=96.87 cores=0-7
/proc/7003/stat : 7003 (TNM) R 7001 7003 7003 0 -1 4202496 5232 0 0 0 9686 1 0 0 25 0 1 0 548438772 474193920 4942 33554432000 134512640 135008748 4292558496 18446744073709551615 134524914 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7003/statm: 115770 4942 46 122 0 114670 0
[pid=7008] ppid=6834 vsize=709632 CPUtime=0 cores=0-7
/proc/7008/stat : 7008 (python) S 6834 6834 6630 0 -1 4202560 916 0 0 0 0 0 0 0 19 0 1 0 548448382 726663168 12940 33554432000 4194304 5683644 140734357152464 18446744073709551615 244446965318 0 0 16781312 514 7030454373666783231 0 0 17 0 0 0 0
/proc/7008/statm: 177408 12940 346 364 0 113227 0
[pid=7010] ppid=7008 vsize=463080 CPUtime=0.76 cores=0-7
/proc/7010/stat : 7010 (TNM) R 7008 7010 7010 0 -1 4202496 5228 0 0 0 75 1 0 0 22 0 1 0 548448383 474193920 4941 33554432000 134512640 135008748 4290297024 18446744073709551615 134527111 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/7010/statm: 115770 4941 46 122 0 114670 0
[pid=6834/tid=6841] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6841/stat : 6841 (python) S 6832 6834 6630 0 -1 4202560 41 934 0 0 0 0 0 0 15 0 35 0 548311578 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 3 0 0 0
[pid=6834/tid=6844] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6844/stat : 6844 (python) S 6832 6834 6630 0 -1 4202560 33 934 0 0 0 0 0 0 15 0 35 0 548311582 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 3 0 0 0
[pid=6834/tid=6847] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6847/stat : 6847 (python) S 6832 6834 6630 0 -1 4202560 39 934 0 0 0 0 0 0 18 0 35 0 548311586 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 1 0 0 0
[pid=6834/tid=6850] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6850/stat : 6850 (python) S 6832 6834 6630 0 -1 4202560 36 934 0 0 0 0 0 0 18 0 35 0 548311590 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 4 0 0 0
[pid=6834/tid=6853] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6853/stat : 6853 (python) S 6832 6834 6630 0 -1 4202560 98 934 0 0 0 0 0 0 18 0 35 0 548311594 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 1 0 0 0
[pid=6834/tid=6856] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6856/stat : 6856 (python) S 6832 6834 6630 0 -1 4202560 39 934 0 0 0 0 0 0 18 0 35 0 548311598 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 6 0 0 0
[pid=6834/tid=6859] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6859/stat : 6859 (python) S 6832 6834 6630 0 -1 4202560 42 934 0 0 0 0 0 0 18 0 35 0 548311602 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 7 0 0 0
[pid=6834/tid=6862] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6862/stat : 6862 (python) S 6832 6834 6630 0 -1 4202560 37 934 0 0 0 0 0 0 18 0 35 0 548311606 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
[pid=6834/tid=6869] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6869/stat : 6869 (python) S 6832 6834 6630 0 -1 4202560 42 934 0 0 0 0 0 0 18 0 35 0 548321624 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
[pid=6834/tid=6884] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6884/stat : 6884 (python) S 6832 6834 6630 0 -1 4202560 64 934 0 0 0 0 0 0 18 0 35 0 548340719 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 2 0 0 0
[pid=6834/tid=6887] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6887/stat : 6887 (python) S 6832 6834 6630 0 -1 4202560 47 934 0 0 0 0 0 0 18 0 35 0 548340724 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 2 0 0 0
[pid=6834/tid=6890] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6890/stat : 6890 (python) S 6832 6834 6630 0 -1 4202560 48 934 0 0 0 0 0 0 18 0 35 0 548340851 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 2 0 0 0
[pid=6834/tid=6894] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6894/stat : 6894 (python) S 6832 6834 6630 0 -1 4202560 34 934 0 0 0 0 0 0 19 0 35 0 548341441 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 7 0 0 0
[pid=6834/tid=6897] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6897/stat : 6897 (python) S 6832 6834 6630 0 -1 4202560 42 934 0 0 0 0 0 0 18 0 35 0 548341447 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 2 0 0 0
[pid=6834/tid=6904] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6904/stat : 6904 (python) S 6832 6834 6630 0 -1 4202560 60 934 0 0 0 0 0 0 18 0 35 0 548350873 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 1 0 0 0
[pid=6834/tid=6908] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6908/stat : 6908 (python) S 6832 6834 6630 0 -1 4202560 90 934 0 0 0 0 0 0 20 0 35 0 548351472 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 2 0 0 0
[pid=6834/tid=6911] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6911/stat : 6911 (python) S 6832 6834 6630 0 -1 4202560 34 934 0 0 0 0 0 0 22 0 35 0 548351479 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
[pid=6834/tid=6918] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6918/stat : 6918 (python) S 6832 6834 6630 0 -1 4202560 40 934 0 0 0 0 0 0 20 0 35 0 548360878 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 1 0 0 0
[pid=6834/tid=6925] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6925/stat : 6925 (python) S 6832 6834 6630 0 -1 4202560 40 934 0 0 0 0 0 0 18 0 35 0 548361499 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 7 0 0 0
[pid=6834/tid=6928] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6928/stat : 6928 (python) S 6832 6834 6630 0 -1 4202560 34 934 0 0 0 0 0 0 21 0 35 0 548361507 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
[pid=6834/tid=6934] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6934/stat : 6934 (python) S 6832 6834 6630 0 -1 4202560 40 934 0 0 0 0 0 0 20 0 35 0 548370694 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 1 0 0 0
[pid=6834/tid=6938] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6938/stat : 6938 (python) S 6832 6834 6630 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 35 0 548371533 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
[pid=6834/tid=6943] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6943/stat : 6943 (python) S 6832 6834 6630 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 35 0 548380125 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 1 0 0 0
[pid=6834/tid=6947] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6947/stat : 6947 (python) S 6832 6834 6630 0 -1 4202560 33 934 0 0 0 0 0 0 19 0 35 0 548380513 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
[pid=6834/tid=6956] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6956/stat : 6956 (python) S 6832 6834 6630 0 -1 4202560 40 934 0 0 0 0 0 0 18 0 35 0 548389628 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 7 0 0 0
[pid=6834/tid=6960] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6960/stat : 6960 (python) S 6832 6834 6630 0 -1 4202560 51 934 0 0 0 0 0 0 18 0 35 0 548390053 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
[pid=6834/tid=6964] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6964/stat : 6964 (python) S 6832 6834 6630 0 -1 4202560 43 934 0 0 0 0 0 0 16 0 35 0 548390541 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 2 0 0 0
[pid=6834/tid=6968] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6968/stat : 6968 (python) S 6832 6834 6630 0 -1 4202560 36 934 0 0 0 0 0 0 18 0 35 0 548391765 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
[pid=6834/tid=6973] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6973/stat : 6973 (python) S 6832 6834 6630 0 -1 4202560 36 934 0 0 0 0 0 0 20 0 35 0 548399239 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
[pid=6834/tid=6986] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6986/stat : 6986 (python) S 6832 6834 6630 0 -1 4202560 34 934 0 0 0 0 0 0 20 0 35 0 548418810 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 1 0 0 0
[pid=6834/tid=6990] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6990/stat : 6990 (python) S 6832 6834 6630 0 -1 4202560 36 934 0 0 0 0 0 0 20 0 35 0 548421531 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
[pid=6834/tid=6996] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/6996/stat : 6996 (python) S 6832 6834 6630 0 -1 4202560 34 934 0 0 0 0 0 0 20 0 35 0 548431562 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 1 0 0 0
[pid=6834/tid=7002] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/7002/stat : 7002 (python) S 6832 6834 6630 0 -1 4202560 33 934 0 0 0 0 0 0 19 0 35 0 548438772 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 0 0 0 0
[pid=6834/tid=7009] ppid=6832 vsize=719876 CPUtime=0 cores=0-7
/proc/6834/task/7009/stat : 7009 (python) S 6832 6834 6630 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 35 0 548448382 737153024 14540 33554432000 4194304 5683644 140734357152464 18446744073709551615 244458768817 0 0 16781312 2 18446604453882608448 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 10911.4
Current children cumulated vsize (KiB) 31761524

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 6834 and gives
#  childrusage.ru_utime.tv_sec=3460
#  childrusage.ru_utime.tv_usec=632903
#  childrusage.ru_stime.tv_sec=9
#  childrusage.ru_stime.tv_usec=954486
# CPU time returned by wait4() is 3470.59
# while last known CPU time is 10911.4
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1375.11
CPU time (s): 10911.4
CPU user time (s): 10900.8
CPU system time (s): 10.63
CPU usage (%): 793.496
Max. virtual memory (cumulated for all children) (KiB): 31761524

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3460.63
system time used= 9.95449
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 151088
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 13138
involuntary context switches= 24461

runsolver used 11.6022 second user time and 30.9763 second system time

The end

Launcher Data 

Begin job on node105 at 2011-05-02 22:11:45
IDJOB=3321780
IDBENCH=82558
IDSOLVER=1712
FILE ID=node105/3321780-1304367105
RUNJOBID= node105-1304366469-6644
PBS_JOBID= 13181765
Free space on /tmp= 73664 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v30000-c126000-S913491256-073-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321780-1304367105/watcher-3321780-1304367105 -o /tmp/evaluation-result-3321780-1304367105/solver-3321780-1304367105 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321780-1304367105.cnf -seed 1155635814 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 785309e098f4d0ba8da3027c8426e931
RANDOM SEED=1155635814

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18079408 kB
Buffers:          8300 kB
Cached:         202480 kB
SwapCached:      26824 kB
Active:        3122012 kB
Inactive:     11558048 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18079408 kB
SwapTotal:    67111528 kB
SwapFree:     62859076 kB
Dirty:           37292 kB
Writeback:           0 kB
AnonPages:    14445544 kB
Mapped:          15564 kB
Slab:            82632 kB
PageTables:      46464 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 20952216 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73632 MiB
End job on node105 at 2011-05-02 22:34:42