Trace number 3385328

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 10199.5 1282.71

General information on the benchmark

Namerandom/large/
unif-k7-r85-v200-c17000-S771170845-016-UNKNOWN.cnf
MD5SUM8a262b49dcf27b27fb405f8a8cdf125f
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2600.06
Satisfiable
(Un)Satisfiability was proved
Number of variables200
Number of clauses17000
Sum of the clauses size119000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 517000

Solver Data 

0.36/0.68	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.36/0.68	c INFO: solving HOME/instance-3385328-1305565129.cnf
0.36/0.68	c INFO: computing task features
0.53/0.80	c INFO: computed variable-clause graph statistics
0.53/0.80	c INFO: computed clause balance statistics
1.13/1.42	c INFO: computed variable balance statistics
1.33/1.66	c INFO: computed small-clause counts
1.33/1.66	c INFO: computed Horn-clause counts
1.33/1.68	c INFO: computed variable graph statistics
1.33/1.68	c INFO: collected features for HOME/instance-3385328-1305565129.cnf in 0.99 s
1.43/1.71	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.35)
1.43/1.74	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.12)
1.43/1.77	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.08)
1.54/1.80	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.06)
1.54/1.83	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.05)
1.54/1.86	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.04)
1.54/1.89	c INFO: running FH@0 for 97 with 4998 remaining (b = 0.03)
1.63/1.93	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.03)
801.63/102.04	c INFO: running adaptg2wsat2009++@0 for 97 with 4898 remaining (b = 0.02)
801.63/102.08	c INFO: running FH@0 for 97 with 4898 remaining (b = 0.02)
802.30/102.12	c INFO: running hybridGM3@0 for 194 with 4898 remaining (b = 0.04)
802.30/102.16	c INFO: running gnovelty+2@0 for 97 with 4898 remaining (b = 0.02)
802.30/102.20	c INFO: running hybridGM3@0 for 194 with 4898 remaining (b = 0.03)
803.06/102.24	c INFO: running hybridGM3@0 for 194 with 4898 remaining (b = 0.03)
803.06/102.29	c INFO: running TNM@0 for 1552 with 4898 remaining (b = 0.20)
803.60/102.34	c INFO: running adaptg2wsat2009++@0 for 97 with 4898 remaining (b = 0.01)
1601.25/202.37	c INFO: running FH@0 for 97 with 4798 remaining (b = 0.01)
1602.00/202.42	c INFO: running hybridGM3@0 for 194 with 4798 remaining (b = 0.02)
1602.52/202.53	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.01)
1603.13/202.70	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.01)
2370.02/298.88	c INFO: running hybridGM3@99 for 97 with 4701 remaining (b = 0.01)
2377.98/299.85	c INFO: running adaptg2wsat2009++@0 for 97 with 4700 remaining (b = 0.01)
2379.58/300.02	c INFO: running adaptg2wsat2009++@100 for 97 with 4700 remaining (b = 0.01)
2393.90/301.83	c INFO: running hybridGM3@99 for 97 with 4698 remaining (b = 0.01)
2401.04/302.71	c INFO: running gnovelty+2@0 for 97 with 4698 remaining (b = 0.01)
2417.78/304.86	c INFO: running hybridGM3@99 for 97 with 4695 remaining (b = 0.01)
3152.54/397.14	c INFO: running hybridGM3@0 for 194 with 4603 remaining (b = 0.02)
3153.21/397.23	c INFO: running hybridGM3@100 for 97 with 4603 remaining (b = 0.01)
3153.86/397.34	c INFO: running hybridGM3@0 for 194 with 4603 remaining (b = 0.02)
3177.06/400.23	c INFO: running TNM@0 for 1455 with 4600 remaining (b = 0.13)
3199.34/403.09	c INFO: running adaptg2wsat2009++@100 for 97 with 4597 remaining (b = 0.01)
3200.93/403.29	c INFO: running FH@0 for 291 with 4597 remaining (b = 0.03)
3223.98/406.19	c INFO: running hybridGM3@193 for 97 with 4594 remaining (b = 0.01)
3952.98/497.66	c INFO: running iPAWS@0 for 97 with 4503 remaining (b = 0.01)
3975.35/500.45	c INFO: running hybridGM3@0 for 194 with 4500 remaining (b = 0.02)
3984.09/501.54	c INFO: running hybridGM3@293 for 97 with 4499 remaining (b = 0.01)
4712.95/593.03	c INFO: running hybridGM3@100 for 97 with 4407 remaining (b = 0.01)
4719.33/593.85	c INFO: running adaptg2wsat2009++@299 for 291 with 4406 remaining (b = 0.02)
4744.80/597.03	c INFO: running hybridGM3@0 for 194 with 4403 remaining (b = 0.02)
4791.04/602.89	c INFO: running TNM@0 for 1552 with 4397 remaining (b = 0.12)
5509.50/693.18	c INFO: running FH@100 for 291 with 4307 remaining (b = 0.02)
5519.03/694.38	c INFO: running hybridGM3@297 for 194 with 4306 remaining (b = 0.02)
5566.00/700.22	c INFO: running iPAWS@0 for 97 with 4300 remaining (b = 0.01)
6302.08/792.76	c INFO: running hybridGM3@91 for 97 with 4207 remaining (b = 0.01)
6412.67/806.61	c INFO: running hybridGM3@97 for 97 with 4194 remaining (b = 0.01)
7026.97/883.84	c INFO: running adaptg2wsat2009++@398 for 194 with 4116 remaining (b = 0.02)
7043.66/885.98	c INFO: running adaptg2wsat2009++@99 for 97 with 4114 remaining (b = 0.01)
7077.09/890.11	c INFO: running hybridGM3@193 for 97 with 4110 remaining (b = 0.01)
7186.86/903.96	c INFO: running hybridGM3@387 for 194 with 4096 remaining (b = 0.02)
7818.80/983.34	c INFO: running FH@0 for 97 with 4017 remaining (b = 0.01)
7835.41/985.45	c INFO: running NCVWr@0 for 485 with 4015 remaining (b = 0.03)
7856.86/988.14	c INFO: running FH@200 for 194 with 4012 remaining (b = 0.01)
8599.28/1081.45	c INFO: running iPAWS@0 for 97 with 3919 remaining (b = 0.01)
8601.40/1081.74	c INFO: running adaptg2wsat2009++@99 for 97 with 3918 remaining (b = 0.01)
8759.17/1101.60	c INFO: running hybridGM3@484 for 97 with 3899 remaining (b = 0.01)
9375.86/1179.10	c INFO: running adaptg2wsat2009++@293 for 291 with 3821 remaining (b = 0.02)
9398.09/1181.81	c INFO: running gnovelty+2@0 for 97 with 3818 remaining (b = 0.01)
9476.87/1191.78	c INFO: running hybridGM3@0 for 194 with 3808 remaining (b = 0.01)
9565.96/1202.96	c INFO: running FH@99 for 291 with 3797 remaining (b = 0.02)
10196.87/1282.21	c INFO: running hybridGM3@0 for 194 with 3718 remaining (b = 0.01)
10199.46/1282.61	c INFO: running TNM@0 for 1552 with 3718 remaining (b = 0.08)
10199.46/1282.67	c INFO: running NCVWr@0 for 485 with 3718 remaining (b = 0.03)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385328-1305565129/watcher-3385328-1305565129 -o /tmp/evaluation-result-3385328-1305565129/solver-3385328-1305565129 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385328-1305565129.cnf -seed 1655447407 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.47 3.27 1.24 1/365 12374
/proc/meminfo: memFree=23358864/32951124 swapFree=44485784/67111528
[pid=12374] ppid=12372 vsize=15780 CPUtime=0 cores=0-7
/proc/12374/stat : 12374 (python) R 12372 12374 12176 0 -1 4202496 902 0 0 0 0 0 0 0 17 0 1 0 668113373 16158720 783 33554432000 4194304 5683644 140735907694928 18446744073709551615 4426576 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/12374/statm: 3945 783 345 364 0 432 0

[startup+0.064611 s]
/proc/loadavg: 7.47 3.27 1.24 1/365 12374
/proc/meminfo: memFree=23358864/32951124 swapFree=44485784/67111528
[pid=12374] ppid=12372 vsize=29792 CPUtime=0.03 cores=0-7
/proc/12374/stat : 12374 (python) R 12372 12374 12176 0 -1 4202496 1460 0 0 0 3 0 0 0 17 0 1 0 668113373 30507008 1300 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446943232 0 2147483391 16781312 2 0 0 0 17 4 0 0 0
/proc/12374/statm: 7448 1300 443 364 0 818 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 29792

[startup+0.1006 s]
/proc/loadavg: 7.47 3.27 1.24 1/365 12374
/proc/meminfo: memFree=23358864/32951124 swapFree=44485784/67111528
[pid=12374] ppid=12372 vsize=85624 CPUtime=0.05 cores=0-7
/proc/12374/stat : 12374 (python) R 12372 12374 12176 0 -1 4202496 2630 0 0 0 4 1 0 0 17 0 1 0 668113373 87678976 2348 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446943232 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/12374/statm: 21406 2348 713 364 0 1623 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 85624

[startup+0.300613 s]
/proc/loadavg: 7.47 3.27 1.24 1/365 12374
/proc/meminfo: memFree=23358864/32951124 swapFree=44485784/67111528
[pid=12374] ppid=12372 vsize=135752 CPUtime=0.19 cores=0-7
/proc/12374/stat : 12374 (python) R 12372 12374 12176 0 -1 4202496 5960 931 1 0 14 5 0 0 18 0 1 0 668113373 139010048 4151 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446941189 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/12374/statm: 33938 4151 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135752

[startup+0.700486 s]
/proc/loadavg: 7.47 3.27 1.24 1/365 12374
/proc/meminfo: memFree=23358864/32951124 swapFree=44485784/67111528
[pid=12374] ppid=12372 vsize=279000 CPUtime=0.43 cores=0-7
/proc/12374/stat : 12374 (python) R 12372 12374 12176 0 -1 4202496 9389 931 1 0 31 12 0 0 17 0 1 0 668113373 285696000 7075 33554432000 4194304 5683644 140735907694928 18446744073709551615 4602978 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/12374/statm: 69750 7075 1761 364 0 5674 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 279000

[startup+1.50035 s]
/proc/loadavg: 7.47 3.27 1.24 2/366 12379
/proc/meminfo: memFree=23333464/32951124 swapFree=44485784/67111528
[pid=12374] ppid=12372 vsize=282588 CPUtime=1.23 cores=0-7
/proc/12374/stat : 12374 (python) R 12372 12374 12176 0 -1 4202496 11325 931 1 0 111 12 0 0 18 0 1 0 668113373 289370112 8059 33554432000 4194304 5683644 140735907694928 18446744073709551615 4943760 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/12374/statm: 70647 8059 1827 364 0 6571 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 282588

[startup+3.101 s]
/proc/loadavg: 7.47 3.27 1.24 9/390 12403
/proc/meminfo: memFree=23287864/32951124 swapFree=44485784/67111528
[pid=12374] ppid=12372 vsize=427072 CPUtime=1.66 cores=0-7
/proc/12374/stat : 12374 (python) S 12372 12374 12176 0 -1 4202496 17242 931 1 0 153 13 0 0 25 0 9 0 668113373 437321728 7525 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/12374/statm: 106768 7525 1891 364 0 42665 0
[pid=12380] ppid=12374 vsize=279232 CPUtime=0 cores=0-7
/proc/12380/stat : 12380 (python) S 12374 12374 12176 0 -1 4202560 736 0 0 0 0 0 0 0 18 0 1 0 668113544 285933568 5714 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965263 0 0 16781312 514 1078410354666471684 0 0 17 1 0 0 0
/proc/12380/statm: 69808 5714 354 364 0 5726 0
[pid=12382] ppid=12380 vsize=1018616 CPUtime=1.38 cores=0-7
/proc/12382/stat : 12382 (gnovelty+2) R 12380 12382 12382 0 -1 4202496 764 0 0 0 138 0 0 0 25 0 1 0 668113545 1043062784 500 33554432000 134512640 135663704 4287033504 18446744073709551615 134524868 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/12382/statm: 254654 500 113 282 0 254370 0
[pid=12383] ppid=12374 vsize=289624 CPUtime=0 cores=0-7
/proc/12383/stat : 12383 (python) S 12374 12374 12176 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 668113547 296574976 5762 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 7422429444335173890 0 0 17 1 0 0 0
/proc/12383/statm: 72406 5762 348 364 0 8321 0
[pid=12385] ppid=12383 vsize=1018616 CPUtime=1.35 cores=0-7
/proc/12385/stat : 12385 (gnovelty+2) R 12383 12385 12385 0 -1 4202496 765 0 0 0 135 0 0 0 25 0 1 0 668113548 1043062784 500 33554432000 134512640 135663704 4287007632 18446744073709551615 134525302 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/12385/statm: 254654 500 113 282 0 254370 0
[pid=12386] ppid=12374 vsize=365548 CPUtime=0 cores=0-7
/proc/12386/stat : 12386 (python) S 12374 12374 12176 0 -1 4202560 746 0 0 0 0 0 0 0 18 0 1 0 668113550 374321152 5785 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 7422425596044509183 0 0 17 6 0 0 0
/proc/12386/statm: 91387 5785 348 364 0 27299 0
[pid=12388] ppid=12386 vsize=418500 CPUtime=1.32 cores=0-7
/proc/12388/stat : 12388 (adaptg2wsat2009) R 12386 12388 12388 0 -1 4202496 1080 0 0 0 132 0 0 0 25 0 1 0 668113551 428544000 809 33554432000 134512640 135107411 4293857328 18446744073709551615 134523432 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12388/statm: 104625 809 49 146 0 103501 0
[pid=12389] ppid=12374 vsize=375804 CPUtime=0 cores=0-7
/proc/12389/stat : 12389 (python) S 12374 12374 12176 0 -1 4202560 713 0 0 0 0 0 0 0 18 0 1 0 668113554 384823296 5960 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 406666474455826431 0 0 17 0 0 0 0
/proc/12389/statm: 93951 5960 348 364 0 29860 0
[pid=12391] ppid=12389 vsize=418500 CPUtime=1.28 cores=0-7
/proc/12391/stat : 12391 (adaptg2wsat2009) R 12389 12391 12391 0 -1 4202496 1066 0 0 0 128 0 0 0 25 0 1 0 668113554 428544000 810 33554432000 134512640 135107411 4293210704 18446744073709551615 134523300 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/12391/statm: 104625 810 49 146 0 103501 0
[pid=12392] ppid=12374 vsize=386060 CPUtime=0 cores=0-7
/proc/12392/stat : 12392 (python) S 12374 12374 12176 0 -1 4202560 721 0 0 0 0 0 0 0 18 0 1 0 668113557 395325440 5917 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 406670322746523647 0 0 17 1 0 0 0
/proc/12392/statm: 96515 5917 348 364 0 32421 0
[pid=12394] ppid=12392 vsize=418500 CPUtime=1.24 cores=0-7
/proc/12394/stat : 12394 (adaptg2wsat2009) R 12392 12394 12394 0 -1 4202496 1068 0 0 0 124 0 0 0 25 0 1 0 668113557 428544000 808 33554432000 134512640 135107411 4292152320 18446744073709551615 134523438 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12394/statm: 104625 808 49 146 0 103501 0
[pid=12395] ppid=12374 vsize=396316 CPUtime=0 cores=0-7
/proc/12395/stat : 12395 (python) S 12374 12374 12176 0 -1 4202560 730 0 0 0 0 0 0 0 18 0 1 0 668113560 405827584 5953 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 6366964802123694338 0 0 17 4 0 0 0
/proc/12395/statm: 99079 5953 348 364 0 34982 0
[pid=12397] ppid=12395 vsize=1018616 CPUtime=1.22 cores=0-7
/proc/12397/stat : 12397 (gnovelty+2) R 12395 12397 12397 0 -1 4202496 759 0 0 0 122 0 0 0 25 0 1 0 668113560 1043062784 501 33554432000 134512640 135663704 4290804032 18446744073709551615 134525213 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12397/statm: 254654 501 113 282 0 254370 0
[pid=12398] ppid=12374 vsize=406572 CPUtime=0 cores=0-7
/proc/12398/stat : 12398 (python) S 12374 12374 12176 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 668113563 416329728 5974 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 7778793807281127423 0 0 17 0 0 0 0
/proc/12398/statm: 101643 5974 348 364 0 37543 0
[pid=12400] ppid=12398 vsize=246900 CPUtime=1.19 cores=0-7
/proc/12400/stat : 12400 (FH) R 12398 12400 12400 0 -1 4202496 937 0 0 0 119 0 0 0 25 0 1 0 668113563 252825600 678 33554432000 134512640 134936988 4294504768 18446744073709551615 134523517 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/12400/statm: 61725 678 37 104 0 61229 0
[pid=12401] ppid=12374 vsize=416828 CPUtime=0 cores=0-7
/proc/12401/stat : 12401 (python) S 12374 12374 12176 0 -1 4202560 770 0 0 0 0 0 0 0 18 0 1 0 668113566 426831872 5984 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 4606908919577575423 0 0 17 4 0 0 0
/proc/12401/statm: 104207 5984 354 364 0 40104 0
[pid=12403] ppid=12401 vsize=418500 CPUtime=1.14 cores=0-7
/proc/12403/stat : 12403 (adaptg2wsat2009) R 12401 12403 12403 0 -1 4202496 1076 0 0 0 114 0 0 0 25 0 1 0 668113567 428544000 809 33554432000 134512640 135107411 4292375024 18446744073709551615 134523503 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/12403/statm: 104625 809 49 146 0 103501 0
[pid=12374/tid=12381] ppid=12372 vsize=427072 CPUtime=0 cores=0-7
/proc/12374/task/12381/stat : 12381 (python) S 12372 12374 12176 0 -1 4202560 26 931 0 0 0 0 0 0 19 0 9 0 668113545 437321728 7525 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 2 0 0 0
[pid=12374/tid=12384] ppid=12372 vsize=427072 CPUtime=0 cores=0-7
/proc/12374/task/12384/stat : 12384 (python) S 12372 12374 12176 0 -1 4202560 21 931 0 0 0 0 0 0 18 0 9 0 668113547 437321728 7525 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 3 0 0 0
[pid=12374/tid=12387] ppid=12372 vsize=427072 CPUtime=0 cores=0-7
/proc/12374/task/12387/stat : 12387 (python) S 12372 12374 12176 0 -1 4202560 25 931 0 0 0 0 0 0 18 0 9 0 668113551 437321728 7525 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 1 0 0 0
[pid=12374/tid=12390] ppid=12372 vsize=427072 CPUtime=0 cores=0-7
/proc/12374/task/12390/stat : 12390 (python) S 12372 12374 12176 0 -1 4202560 92 931 0 0 0 0 0 0 23 0 9 0 668113554 437321728 7525 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 0 0 0 0
[pid=12374/tid=12393] ppid=12372 vsize=427072 CPUtime=0 cores=0-7
/proc/12374/task/12393/stat : 12393 (python) S 12372 12374 12176 0 -1 4202560 33 931 0 0 0 0 0 0 19 0 9 0 668113557 437321728 7525 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 0 0 0 0
[pid=12374/tid=12396] ppid=12372 vsize=427072 CPUtime=0 cores=0-7
/proc/12374/task/12396/stat : 12396 (python) S 12372 12374 12176 0 -1 4202560 33 931 0 0 0 0 0 0 21 0 9 0 668113560 437321728 7525 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 4 0 0 0
[pid=12374/tid=12399] ppid=12372 vsize=427072 CPUtime=0 cores=0-7
/proc/12374/task/12399/stat : 12399 (python) S 12372 12374 12176 0 -1 4202560 35 931 0 0 0 0 0 0 23 0 9 0 668113563 437321728 7525 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 0 0 0 0
[pid=12374/tid=12402] ppid=12372 vsize=427072 CPUtime=0 cores=0-7
/proc/12374/task/12402/stat : 12402 (python) S 12372 12374 12176 0 -1 4202560 34 931 0 0 0 0 0 0 25 0 9 0 668113566 437321728 7525 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.78
Current children cumulated vsize (KiB) 8319804

[startup+6.30141 s]
/proc/loadavg: 7.51 3.35 1.28 9/390 12404
/proc/meminfo: memFree=23284764/32951124 swapFree=44485784/67111528
[pid=12374] ppid=12372 vsize=427072 CPUtime=1.66 cores=0-7
/proc/12374/stat : 12374 (python) S 12372 12374 12176 0 -1 4202496 17242 931 1 0 153 13 0 0 25 0 9 0 668113373 437321728 7525 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/12374/statm: 106768 7525 1891 364 0 42665 0
[pid=12380] ppid=12374 vsize=279232 CPUtime=0.01 cores=0-7
/proc/12380/stat : 12380 (python) S 12374 12374 12176 0 -1 4202560 898 0 0 0 0 1 0 0 18 0 1 0 668113544 285933568 5737 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965263 0 0 16781312 514 1078410354666471684 0 0 17 1 0 0 0
/proc/12380/statm: 69808 5737 377 364 0 5726 0
[pid=12382] ppid=12380 vsize=1018616 CPUtime=4.58 cores=0-7
/proc/12382/stat : 12382 (gnovelty+2) R 12380 12382 12382 0 -1 4202496 764 0 0 0 458 0 0 0 25 0 1 0 668113545 1043062784 500 33554432000 134512640 135663704 4287033504 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/12382/statm: 254654 500 113 282 0 254370 0
[pid=12383] ppid=12374 vsize=289624 CPUtime=0.01 cores=0-7
/proc/12383/stat : 12383 (python) S 12374 12374 12176 0 -1 4202560 906 0 0 0 0 1 0 0 18 0 1 0 668113547 296574976 5785 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 7422429444335173890 0 0 17 1 0 0 0
/proc/12383/statm: 72406 5785 371 364 0 8321 0
[pid=12385] ppid=12383 vsize=1018616 CPUtime=4.55 cores=0-7
/proc/12385/stat : 12385 (gnovelty+2) R 12383 12385 12385 0 -1 4202496 765 0 0 0 455 0 0 0 25 0 1 0 668113548 1043062784 500 33554432000 134512640 135663704 4287007632 18446744073709551615 134524518 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/12385/statm: 254654 500 113 282 0 254370 0
[pid=12386] ppid=12374 vsize=365548 CPUtime=0 cores=0-7
/proc/12386/stat : 12386 (python) S 12374 12374 12176 0 -1 4202560 901 0 0 0 0 0 0 0 18 0 1 0 668113550 374321152 5808 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 7422425596044476674 0 0 17 6 0 0 0
/proc/12386/statm: 91387 5808 371 364 0 27299 0
[pid=12388] ppid=12386 vsize=418636 CPUtime=4.52 cores=0-7
/proc/12388/stat : 12388 (adaptg2wsat2009) R 12386 12388 12388 0 -1 4202496 1227 0 0 0 452 0 0 0 25 0 1 0 668113551 428683264 956 33554432000 134512640 135107411 4293857328 18446744073709551615 134532349 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12388/statm: 104659 956 59 146 0 103535 0
[pid=12389] ppid=12374 vsize=375804 CPUtime=0.01 cores=0-7
/proc/12389/stat : 12389 (python) S 12374 12374 12176 0 -1 4202560 789 0 0 0 0 1 0 0 18 0 1 0 668113554 384823296 5983 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 406666474455793923 0 0 17 0 0 0 0
/proc/12389/statm: 93951 5983 371 364 0 29860 0
[pid=12391] ppid=12389 vsize=418636 CPUtime=4.46 cores=0-7
/proc/12391/stat : 12391 (adaptg2wsat2009) R 12389 12391 12391 0 -1 4202496 1213 0 0 0 446 0 0 0 25 0 1 0 668113554 428683264 957 33554432000 134512640 135107411 4293210704 18446744073709551615 134532408 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/12391/statm: 104659 957 59 146 0 103535 0
[pid=12392] ppid=12374 vsize=386060 CPUtime=0.01 cores=0-7
/proc/12392/stat : 12392 (python) S 12374 12374 12176 0 -1 4202560 798 0 0 0 0 1 0 0 18 0 1 0 668113557 395325440 5940 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 406670322746491139 0 0 17 1 0 0 0
/proc/12392/statm: 96515 5940 371 364 0 32421 0
[pid=12394] ppid=12392 vsize=418636 CPUtime=4.4 cores=0-7
/proc/12394/stat : 12394 (adaptg2wsat2009) R 12392 12394 12394 0 -1 4202496 1214 0 0 0 440 0 0 0 25 0 1 0 668113557 428683264 954 33554432000 134512640 135107411 4292152320 18446744073709551615 134532242 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12394/statm: 104659 954 59 146 0 103535 0
[pid=12395] ppid=12374 vsize=396316 CPUtime=0 cores=0-7
/proc/12395/stat : 12395 (python) S 12374 12374 12176 0 -1 4202560 805 0 0 0 0 0 0 0 18 0 1 0 668113560 405827584 5976 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 6366964802123694338 0 0 17 4 0 0 0
/proc/12395/statm: 99079 5976 371 364 0 34982 0
[pid=12397] ppid=12395 vsize=1018616 CPUtime=4.37 cores=0-7
/proc/12397/stat : 12397 (gnovelty+2) R 12395 12397 12397 0 -1 4202496 759 0 0 0 437 0 0 0 25 0 1 0 668113560 1043062784 501 33554432000 134512640 135663704 4290804032 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12397/statm: 254654 501 113 282 0 254370 0
[pid=12398] ppid=12374 vsize=406572 CPUtime=0 cores=0-7
/proc/12398/stat : 12398 (python) S 12374 12374 12176 0 -1 4202560 820 0 0 0 0 0 0 0 18 0 1 0 668113563 416329728 5996 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 7778793807281094916 0 0 17 0 0 0 0
/proc/12398/statm: 101643 5996 370 364 0 37543 0
[pid=12400] ppid=12398 vsize=248648 CPUtime=4.36 cores=0-7
/proc/12400/stat : 12400 (FH) R 12398 12400 12400 0 -1 4202496 1117 0 0 0 436 0 0 0 25 0 1 0 668113563 254615552 856 33554432000 134512640 134936988 4294504768 18446744073709551615 134552382 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/12400/statm: 62162 856 41 104 0 61666 0
[pid=12401] ppid=12374 vsize=416828 CPUtime=0 cores=0-7
/proc/12401/stat : 12401 (python) S 12374 12374 12176 0 -1 4202560 857 0 0 0 0 0 0 0 18 0 1 0 668113566 426831872 6006 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 4606908919577542916 0 0 17 4 0 0 0
/proc/12401/statm: 104207 6006 376 364 0 40104 0
[pid=12403] ppid=12401 vsize=418636 CPUtime=4.31 cores=0-7
/proc/12403/stat : 12403 (adaptg2wsat2009) R 12401 12403 12403 0 -1 4202496 1223 0 0 0 431 0 0 0 25 0 1 0 668113567 428683264 956 33554432000 134512640 135107411 4292375024 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/12403/statm: 104659 956 59 146 0 103535 0

################
# More data... #
################

[pid=12480/tid=12498] ppid=12374 vsize=550156 CPUtime=0 cores=0-7
/proc/12480/task/12498/stat : 12498 (python) S 12374 12374 12176 0 -1 4202560 82 0 0 0 0 0 0 0 18 0 2 0 668172671 563359744 6358 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 514 18446604461235795968 0 0 -1 1 0 0 0
[pid=12483] ppid=12374 vsize=560412 CPUtime=2.74 cores=0-7
/proc/12483/stat : 12483 (python) S 12374 12374 12176 0 -1 4202560 985 0 0 0 143 131 0 0 18 0 1 0 668153396 573861888 6301 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 212724713649045768 0 0 17 5 0 0 0
/proc/12483/statm: 140103 6301 379 364 0 75958 0
[pid=12485] ppid=12483 vsize=457124 CPUtime=876.1 cores=0-7
/proc/12485/stat : 12485 (TNM) R 12483 12485 12485 0 -1 4202496 1119 0 0 0 87610 0 0 0 25 0 1 0 668153397 468094976 837 33554432000 134512640 135008748 4289738000 18446744073709551615 134527524 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/12485/statm: 114281 837 46 122 0 113181 0
[pid=12490] ppid=12374 vsize=570668 CPUtime=0.3 cores=0-7
/proc/12490/stat : 12490 (python) S 12374 12374 12176 0 -1 4202560 1187 0 0 0 17 13 0 0 19 0 2 0 668163140 584364032 6295 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/12490/statm: 142667 6295 408 364 0 78519 0
[pid=12492] ppid=12490 vsize=335996 CPUtime=98.93 cores=0-7
/proc/12492/stat : 12492 (iPAWS) T 12490 12492 12492 0 -1 4202496 1603 0 0 0 9893 0 0 0 25 0 1 0 668163140 344059904 1320 33554432000 134512640 135114423 4294794320 18446744073709551615 134514774 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/12492/statm: 83999 1320 59 147 0 83851 0
[pid=12490/tid=12500] ppid=12374 vsize=570668 CPUtime=0 cores=0-7
/proc/12490/task/12500/stat : 12500 (python) S 12374 12374 12176 0 -1 4202560 87 0 0 0 0 0 0 0 19 0 2 0 668173071 584364032 6295 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 514 18446604457219871104 0 0 -1 7 0 0 0
[pid=12493] ppid=12374 vsize=580924 CPUtime=1.2 cores=0-7
/proc/12493/stat : 12493 (python) S 12374 12374 12176 0 -1 4202560 1192 0 0 0 63 57 0 0 18 0 2 0 668163419 594866176 6360 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/12493/statm: 145231 6360 407 364 0 81080 0
[pid=12495] ppid=12493 vsize=268424 CPUtime=389.79 cores=0-7
/proc/12495/stat : 12495 (hybridGM3) T 12493 12495 12495 0 -1 4202496 5183 0 0 0 38979 0 0 0 25 0 1 0 668163419 274866176 4897 33554432000 134512640 135204385 4291139760 18446744073709551615 134538541 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/12495/statm: 67106 4897 63 169 0 66936 0
[pid=12493/tid=12510] ppid=12374 vsize=580924 CPUtime=0 cores=0-7
/proc/12493/task/12510/stat : 12510 (python) S 12374 12374 12176 0 -1 4202560 84 0 0 0 0 0 0 0 18 0 2 0 668183389 594866176 6360 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 514 18446604461227930048 0 0 -1 2 0 0 0
[pid=12501] ppid=12374 vsize=591180 CPUtime=1.83 cores=0-7
/proc/12501/stat : 12501 (python) S 12374 12374 12176 0 -1 4202560 1191 0 0 0 96 87 0 0 18 0 2 0 668173077 605368320 6300 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/12501/statm: 147795 6300 401 364 0 83641 0
[pid=12503] ppid=12501 vsize=268420 CPUtime=587.14 cores=0-7
/proc/12503/stat : 12503 (hybridGM3) T 12501 12503 12503 0 -1 4202496 5182 0 0 0 58713 1 0 0 25 0 1 0 668173077 274862080 4896 33554432000 134512640 135204385 4287976448 18446744073709551615 134537891 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/12503/statm: 67105 4896 63 169 0 66935 0
[pid=12501/tid=12516] ppid=12374 vsize=591180 CPUtime=0 cores=0-7
/proc/12501/task/12516/stat : 12516 (python) S 12374 12374 12176 0 -1 4202560 91 0 0 0 0 0 0 0 18 0 2 0 668192643 605368320 6300 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 514 18446604465405654208 0 0 -1 5 0 0 0
[pid=12504] ppid=12374 vsize=601436 CPUtime=2.07 cores=0-7
/proc/12504/stat : 12504 (python) S 12374 12374 12176 0 -1 4202560 990 0 0 0 100 107 0 0 18 0 1 0 668173663 615870464 6326 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 212724713649045765 0 0 17 6 0 0 0
/proc/12504/statm: 150359 6326 373 364 0 86202 0
[pid=12506] ppid=12504 vsize=457124 CPUtime=671.05 cores=0-7
/proc/12506/stat : 12506 (TNM) R 12504 12506 12506 0 -1 4202496 1119 0 0 0 67105 0 0 0 25 0 1 0 668173663 468094976 837 33554432000 134512640 135008748 4288987232 18446744073709551615 134527313 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12506/statm: 114281 837 46 122 0 113181 0
[pid=12511] ppid=12374 vsize=611692 CPUtime=0.32 cores=0-7
/proc/12511/stat : 12511 (python) S 12374 12374 12176 0 -1 4202560 1182 0 0 0 16 16 0 0 19 0 2 0 668183395 626372608 6313 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/12511/statm: 152923 6313 402 364 0 88763 0
[pid=12513] ppid=12511 vsize=335996 CPUtime=101.85 cores=0-7
/proc/12513/stat : 12513 (iPAWS) T 12511 12513 12513 0 -1 4202496 1602 0 0 0 10185 0 0 0 25 0 1 0 668183397 344059904 1318 33554432000 134512640 135114423 4288106016 18446744073709551615 134517052 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/12513/statm: 83999 1318 59 147 0 83851 0
[pid=12511/tid=12518] ppid=12374 vsize=611692 CPUtime=0 cores=0-7
/proc/12511/task/12518/stat : 12518 (python) S 12374 12374 12176 0 -1 4202560 87 0 0 0 0 0 0 0 19 0 2 0 668194028 626372608 6313 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 514 18446604466415254208 0 0 -1 5 0 0 0
[pid=12524] ppid=12374 vsize=621948 CPUtime=0.31 cores=0-7
/proc/12524/stat : 12524 (python) S 12374 12374 12176 0 -1 4202560 1199 0 0 0 17 14 0 0 19 0 2 0 668211707 636874752 6341 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/12524/statm: 155487 6341 402 364 0 91324 0
[pid=12526] ppid=12524 vsize=248648 CPUtime=98.25 cores=0-7
/proc/12526/stat : 12526 (FH) T 12524 12526 12526 0 -1 4202496 1142 0 0 0 9825 0 0 0 25 0 1 0 668211708 254615552 857 33554432000 134512640 134936988 4287840256 18446744073709551615 134552080 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/12526/statm: 62162 857 41 104 0 61666 0
[pid=12524/tid=12535] ppid=12374 vsize=621948 CPUtime=0 cores=0-7
/proc/12524/task/12535/stat : 12535 (python) S 12374 12374 12176 0 -1 4202560 93 0 0 0 0 0 0 0 19 0 2 0 668221541 636874752 6341 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 514 18446604469249532160 0 0 -1 7 0 0 0
[pid=12527] ppid=12374 vsize=632204 CPUtime=0.95 cores=0-7
/proc/12527/stat : 12527 (python) S 12374 12374 12176 0 -1 4202560 1086 0 0 0 47 48 0 0 18 0 1 0 668211918 647376896 6409 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 212724713649045766 0 0 17 7 0 0 0
/proc/12527/statm: 158051 6409 379 364 0 93885 0
[pid=12529] ppid=12527 vsize=418456 CPUtime=296.26 cores=0-7
/proc/12529/stat : 12529 (NCVWr) R 12527 12529 12529 0 -1 4202496 1158 0 0 0 29625 1 0 0 25 0 1 0 668211919 428498944 873 33554432000 134512640 135113558 4286859008 18446744073709551615 134531124 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/12529/statm: 104614 873 58 147 0 103489 0
[pid=12532] ppid=12374 vsize=642460 CPUtime=0.3 cores=0-7
/proc/12532/stat : 12532 (python) S 12374 12374 12176 0 -1 4202560 1244 0 0 0 16 14 0 0 19 0 2 0 668221519 657879040 6462 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/12532/statm: 160615 6462 408 364 0 96446 0
[pid=12534] ppid=12532 vsize=335996 CPUtime=100.07 cores=0-7
/proc/12534/stat : 12534 (iPAWS) T 12532 12534 12534 0 -1 4202496 1609 0 0 0 10007 0 0 0 25 0 1 0 668221519 344059904 1320 33554432000 134512640 135114423 4290940448 18446744073709551615 134514760 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/12534/statm: 83999 1320 59 147 0 83851 0
[pid=12532/tid=12539] ppid=12374 vsize=642460 CPUtime=0 cores=0-7
/proc/12532/task/12539/stat : 12539 (python) S 12374 12374 12176 0 -1 4202560 91 0 0 0 0 0 0 0 19 0 2 0 668231549 657879040 6462 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 514 18446604450100153152 0 0 -1 6 0 0 0
[pid=12540] ppid=12374 vsize=652716 CPUtime=0.32 cores=0-7
/proc/12540/stat : 12540 (python) S 12374 12374 12176 0 -1 4202560 1256 0 0 0 15 17 0 0 19 0 2 0 668231554 668381184 6402 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/12540/statm: 163179 6402 408 364 0 99007 0
[pid=12542] ppid=12540 vsize=1018616 CPUtime=99.47 cores=0-7
/proc/12542/stat : 12542 (gnovelty+2) T 12540 12542 12542 0 -1 4202496 794 0 0 0 9947 0 0 0 25 0 1 0 668231555 1043062784 501 33554432000 134512640 135663704 4292289360 18446744073709551615 134525255 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/12542/statm: 254654 501 113 282 0 254370 0
[pid=12540/tid=12549] ppid=12374 vsize=652716 CPUtime=0 cores=0-7
/proc/12540/task/12549/stat : 12549 (python) S 12374 12374 12176 0 -1 4202560 92 0 0 0 0 0 0 0 19 0 2 0 668241588 668381184 6402 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 514 18446604447147391808 0 0 -1 5 0 0 0
[pid=12543] ppid=12374 vsize=662972 CPUtime=0.28 cores=0-7
/proc/12543/stat : 12543 (python) S 12374 12374 12176 0 -1 4202560 1058 0 0 0 16 12 0 0 18 0 1 0 668232552 678883328 6467 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 212724713649045764 0 0 17 6 0 0 0
/proc/12543/statm: 165743 6467 378 364 0 101568 0
[pid=12545] ppid=12543 vsize=268424 CPUtime=86.78 cores=0-7
/proc/12545/stat : 12545 (hybridGM3) R 12543 12545 12545 0 -1 4202496 5187 0 0 0 8678 0 0 0 25 0 1 0 668232552 274866176 4897 33554432000 134512640 135204385 4294178752 18446744073709551615 134537792 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12545/statm: 67106 4897 63 169 0 66936 0
[pid=12550] ppid=12374 vsize=673228 CPUtime=0 cores=0-7
/proc/12550/stat : 12550 (python) S 12374 12374 12176 0 -1 4202560 956 0 0 0 0 0 0 0 19 0 1 0 668241594 689385472 6429 33554432000 4194304 5683644 140735907694928 18446744073709551615 244446965318 0 0 16781312 514 3251031862134439935 0 0 17 1 0 0 0
/proc/12550/statm: 168307 6429 356 364 0 104129 0
[pid=12552] ppid=12550 vsize=268428 CPUtime=0.31 cores=0-7
/proc/12552/stat : 12552 (hybridGM3) R 12550 12552 12552 0 -1 4202496 5190 0 0 0 31 0 0 0 21 0 1 0 668241595 274870272 4897 33554432000 134512640 135204385 4286633936 18446744073709551615 134538041 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12552/statm: 67107 4897 63 169 0 66937 0
[pid=12374/tid=12381] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12381/stat : 12381 (python) S 12372 12374 12176 0 -1 4202560 26 931 0 0 0 0 0 0 19 0 34 0 668113545 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 2 0 0 0
[pid=12374/tid=12384] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12384/stat : 12384 (python) S 12372 12374 12176 0 -1 4202560 21 931 0 0 0 0 0 0 18 0 34 0 668113547 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 3 0 0 0
[pid=12374/tid=12387] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12387/stat : 12387 (python) S 12372 12374 12176 0 -1 4202560 40 931 0 0 0 0 0 0 15 0 34 0 668113551 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 1 0 0 0
[pid=12374/tid=12390] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12390/stat : 12390 (python) S 12372 12374 12176 0 -1 4202560 96 931 0 0 0 0 0 0 18 0 34 0 668113554 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 0 0 0 0
[pid=12374/tid=12393] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12393/stat : 12393 (python) S 12372 12374 12176 0 -1 4202560 37 931 0 0 0 0 0 0 18 0 34 0 668113557 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 0 0 0 0
[pid=12374/tid=12396] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12396/stat : 12396 (python) S 12372 12374 12176 0 -1 4202560 33 931 0 0 0 0 0 0 21 0 34 0 668113560 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 4 0 0 0
[pid=12374/tid=12399] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12399/stat : 12399 (python) S 12372 12374 12176 0 -1 4202560 39 931 0 0 0 0 0 0 18 0 34 0 668113563 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 0 0 0 0
[pid=12374/tid=12402] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12402/stat : 12402 (python) S 12372 12374 12176 0 -1 4202560 52 931 0 0 0 0 0 0 18 0 34 0 668113566 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 4 0 0 0
[pid=12374/tid=12411] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12411/stat : 12411 (python) S 12372 12374 12176 0 -1 4202560 50 931 0 0 0 0 0 0 18 0 34 0 668123578 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 3 0 0 0
[pid=12374/tid=12415] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12415/stat : 12415 (python) S 12372 12374 12176 0 -1 4202560 46 931 0 0 0 0 0 0 18 0 34 0 668123582 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 4 0 0 0
[pid=12374/tid=12419] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12419/stat : 12419 (python) S 12372 12374 12176 0 -1 4202560 37 931 0 0 0 0 0 0 18 0 34 0 668123585 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 7 0 0 0
[pid=12374/tid=12423] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12423/stat : 12423 (python) S 12372 12374 12176 0 -1 4202560 35 931 0 0 0 0 0 0 20 0 34 0 668123589 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 1 0 0 0
[pid=12374/tid=12427] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12427/stat : 12427 (python) S 12372 12374 12176 0 -1 4202560 48 931 0 0 0 0 0 0 18 0 34 0 668123594 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 7 0 0 0
[pid=12374/tid=12431] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12431/stat : 12431 (python) S 12372 12374 12176 0 -1 4202560 42 931 0 0 0 0 0 0 18 0 34 0 668123598 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 3 0 0 0
[pid=12374/tid=12434] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12434/stat : 12434 (python) S 12372 12374 12176 0 -1 4202560 33 931 0 0 0 0 0 0 19 0 34 0 668123603 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 4 0 0 0
[pid=12374/tid=12437] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12437/stat : 12437 (python) S 12372 12374 12176 0 -1 4202560 42 931 0 0 0 0 0 0 18 0 34 0 668123608 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 1 0 0 0
[pid=12374/tid=12454] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12454/stat : 12454 (python) S 12372 12374 12176 0 -1 4202560 42 931 0 0 0 0 0 0 18 0 34 0 668133611 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 4 0 0 0
[pid=12374/tid=12457] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12457/stat : 12457 (python) S 12372 12374 12176 0 -1 4202560 40 931 0 0 0 0 0 0 18 0 34 0 668133616 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 1 0 0 0
[pid=12374/tid=12469] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12469/stat : 12469 (python) S 12372 12374 12176 0 -1 4202560 40 931 0 0 0 0 0 0 20 0 34 0 668143645 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 0 0 0 0
[pid=12374/tid=12477] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12477/stat : 12477 (python) S 12372 12374 12176 0 -1 4202560 33 931 0 0 0 0 0 0 19 0 34 0 668153088 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 2 0 0 0
[pid=12374/tid=12481] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12481/stat : 12481 (python) S 12372 12374 12176 0 -1 4202560 47 931 0 0 0 0 0 0 16 0 34 0 668153108 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 1 0 0 0
[pid=12374/tid=12484] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12484/stat : 12484 (python) S 12372 12374 12176 0 -1 4202560 34 931 0 0 0 0 0 0 20 0 34 0 668153397 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 1 0 0 0
[pid=12374/tid=12491] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12491/stat : 12491 (python) S 12372 12374 12176 0 -1 4202560 37 931 0 0 0 0 0 0 20 0 34 0 668163140 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 7 0 0 0
[pid=12374/tid=12494] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12494/stat : 12494 (python) S 12372 12374 12176 0 -1 4202560 39 931 0 0 0 0 0 0 18 0 34 0 668163419 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 7 0 0 0
[pid=12374/tid=12502] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12502/stat : 12502 (python) S 12372 12374 12176 0 -1 4202560 45 931 0 0 0 0 0 0 18 0 34 0 668173077 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 7 0 0 0
[pid=12374/tid=12505] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12505/stat : 12505 (python) S 12372 12374 12176 0 -1 4202560 33 931 0 0 0 0 0 0 20 0 34 0 668173663 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 7 0 0 0
[pid=12374/tid=12512] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12512/stat : 12512 (python) S 12372 12374 12176 0 -1 4202560 37 931 0 0 0 0 0 0 19 0 34 0 668183396 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 7 0 0 0
[pid=12374/tid=12525] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12525/stat : 12525 (python) S 12372 12374 12176 0 -1 4202560 43 931 0 0 0 0 0 0 20 0 34 0 668211708 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 0 0 0 0
[pid=12374/tid=12528] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12528/stat : 12528 (python) S 12372 12374 12176 0 -1 4202560 36 931 0 0 0 0 0 0 20 0 34 0 668211919 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 7 0 0 0
[pid=12374/tid=12533] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12533/stat : 12533 (python) S 12372 12374 12176 0 -1 4202560 37 931 0 0 0 0 0 0 20 0 34 0 668221519 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 1 0 0 0
[pid=12374/tid=12541] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12541/stat : 12541 (python) S 12372 12374 12176 0 -1 4202560 36 931 0 0 0 0 0 0 19 0 34 0 668231554 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 6 0 0 0
[pid=12374/tid=12544] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12544/stat : 12544 (python) S 12372 12374 12176 0 -1 4202560 31 931 0 0 0 0 0 0 20 0 34 0 668232552 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 6 0 0 0
[pid=12374/tid=12551] ppid=12372 vsize=683472 CPUtime=0 cores=0-7
/proc/12374/task/12551/stat : 12551 (python) S 12372 12374 12176 0 -1 4202560 34 931 0 0 0 0 0 0 19 0 34 0 668241595 699875328 8019 33554432000 4194304 5683644 140735907694928 18446744073709551615 244458768817 0 0 16781312 2 18446604450428869312 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 10199.5
Current children cumulated vsize (KiB) 32463764

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 12374 and gives
#  childrusage.ru_utime.tv_sec=2265
#  childrusage.ru_utime.tv_usec=255628
#  childrusage.ru_stime.tv_sec=14
#  childrusage.ru_stime.tv_usec=542789
# CPU time returned by wait4() is 2279.8
# while last known CPU time is 10199.5
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1282.71
CPU time (s): 10199.5
CPU user time (s): 10184
CPU system time (s): 15.48
CPU usage (%): 795.148
Max. virtual memory (cumulated for all children) (KiB): 32463764

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2265.26
system time used= 14.5428
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 89468
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 14739
involuntary context switches= 10631

runsolver used 14.0839 second user time and 31.6452 second system time

The end

Launcher Data 

Begin job on node105 at 2011-05-16 18:58:49
IDJOB=3385328
IDBENCH=82593
IDSOLVER=1870
FILE ID=node105/3385328-1305565129
RUNJOBID= node105-1305564966-12190
PBS_JOBID= 13350145
Free space on /tmp= 73544 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v200-c17000-S771170845-016-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385328-1305565129/watcher-3385328-1305565129 -o /tmp/evaluation-result-3385328-1305565129/solver-3385328-1305565129 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385328-1305565129.cnf -seed 1655447407 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 8a262b49dcf27b27fb405f8a8cdf125f
RANDOM SEED=1655447407

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23359152 kB
Buffers:        320368 kB
Cached:        7151612 kB
SwapCached:      39636 kB
Active:        2218136 kB
Inactive:      7087396 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23359152 kB
SwapTotal:    67111528 kB
SwapFree:     44485784 kB
Dirty:           32208 kB
Writeback:           0 kB
AnonPages:     1801892 kB
Mapped:          15544 kB
Slab:           143212 kB
PageTables:      72532 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 91325432 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73512 MiB
End job on node105 at 2011-05-16 19:20:14