Trace number 3385352

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 8593.24 1086.91

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v40000-c168000-S64788608-008-UNKNOWN.cnf
MD5SUMc0e1e71ade06eb64bd0f5dbaf6b892a1
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark52.1051
Satisfiable
(Un)Satisfiability was proved
Number of variables40000
Number of clauses168000
Sum of the clauses size504000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3168000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.37/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.37/0.67	c INFO: solving HOME/instance-3385352-1305565599.cnf
0.37/0.67	c INFO: computing task features
0.94/1.24	c INFO: computed variable-clause graph statistics
0.94/1.27	c INFO: computed clause balance statistics
3.85/4.17	c INFO: computed variable balance statistics
6.26/6.53	c INFO: computed small-clause counts
6.26/6.54	c INFO: computed Horn-clause counts
8.25/8.55	c INFO: computed variable graph statistics
8.25/8.55	c INFO: collected features for HOME/instance-3385352-1305565599.cnf in 7.86 s
8.25/8.58	c INFO: running hybridGM3@0 for 97 with 4992 remaining (b = 0.35)
8.35/8.61	c INFO: running hybridGM3@0 for 97 with 4992 remaining (b = 0.21)
8.35/8.64	c INFO: running hybridGM3@0 for 97 with 4992 remaining (b = 0.14)
8.35/8.67	c INFO: running hybridGM3@0 for 97 with 4992 remaining (b = 0.10)
8.35/8.70	c INFO: running hybridGM3@0 for 97 with 4991 remaining (b = 0.07)
8.74/8.74	c INFO: running hybridGM3@0 for 194 with 4991 remaining (b = 0.10)
8.74/8.77	c INFO: running hybridGM3@0 for 194 with 4991 remaining (b = 0.08)
8.74/8.81	c INFO: running hybridGM3@0 for 194 with 4991 remaining (b = 0.06)
710.88/96.79	c INFO: running TNM@0 for 97 with 4903 remaining (b = 0.02)
804.04/108.41	c INFO: running hybridGM3@0 for 97 with 4892 remaining (b = 0.03)
811.21/109.37	c INFO: running hybridGM3@0 for 97 with 4891 remaining (b = 0.03)
811.99/109.44	c INFO: running hybridGM3@0 for 97 with 4891 remaining (b = 0.03)
811.99/109.47	c INFO: running hybridGM3@0 for 97 with 4891 remaining (b = 0.03)
910.04/121.74	c INFO: running hybridGM3@0 for 194 with 4878 remaining (b = 0.04)
979.40/130.49	c INFO: running hybridGM3@0 for 194 with 4870 remaining (b = 0.03)
1511.31/197.16	c INFO: running TNM@0 for 97 with 4803 remaining (b = 0.01)
1580.73/205.85	c INFO: running hybridGM3@0 for 97 with 4794 remaining (b = 0.02)
1580.73/205.89	c INFO: running hybridGM3@88 for 97 with 4794 remaining (b = 0.02)
1581.44/205.92	c INFO: running hybridGM3@0 for 97 with 4794 remaining (b = 0.02)
1603.76/208.78	c INFO: running hybridGM3@100 for 97 with 4791 remaining (b = 0.02)
1620.50/210.83	c INFO: running hybridGM3@10 for 97 with 4789 remaining (b = 0.02)
1757.66/228.07	c INFO: running hybridGM3@0 for 194 with 4772 remaining (b = 0.03)
2274.40/292.87	c INFO: running hybridGM3@0 for 194 with 4707 remaining (b = 0.02)
2311.90/297.56	c INFO: running hybridGM3@0 for 194 with 4703 remaining (b = 0.02)
2380.46/306.17	c INFO: running hybridGM3@32 for 97 with 4694 remaining (b = 0.01)
2389.21/307.21	c INFO: running TNM@0 for 97 with 4693 remaining (b = 0.01)
2396.37/308.17	c INFO: running TNM@0 for 97 with 4692 remaining (b = 0.01)
2481.64/318.89	c INFO: running hybridGM3@16 for 97 with 4681 remaining (b = 0.01)
2551.77/327.64	c INFO: running hybridGM3@100 for 97 with 4673 remaining (b = 0.01)
3141.22/401.54	c INFO: running hybridGM3@134 for 97 with 4599 remaining (b = 0.01)
3188.95/407.60	c INFO: running TNM@0 for 97 with 4593 remaining (b = 0.01)
3229.57/412.67	c INFO: running TNM@0 for 97 with 4588 remaining (b = 0.01)
3242.30/414.25	c INFO: running hybridGM3@0 for 194 with 4586 remaining (b = 0.02)
3311.57/422.99	c INFO: running TNM@0 for 97 with 4577 remaining (b = 0.01)
3329.87/425.29	c INFO: running hybridGM3@83 for 97 with 4575 remaining (b = 0.01)
3658.10/466.48	c INFO: running TNM@0 for 97 with 4534 remaining (b = 0.01)
3883.42/494.74	c INFO: running hybridGM3@94 for 97 with 4505 remaining (b = 0.01)
3894.67/496.18	c INFO: running hybridGM3@100 for 97 with 4504 remaining (b = 0.01)
3952.72/503.46	c INFO: running hybridGM3@0 for 194 with 4497 remaining (b = 0.02)
3980.57/507.01	c INFO: running TNM@194 for 1843 with 4493 remaining (b = 0.15)
4029.14/513.07	c INFO: running hybridGM3@0 for 194 with 4487 remaining (b = 0.02)
4111.17/523.39	c INFO: running adaptg2wsat2009++@0 for 194 with 4477 remaining (b = 0.02)
4457.55/566.89	c INFO: running hybridGM3@0 for 194 with 4433 remaining (b = 0.01)
4614.37/586.54	c INFO: running hybridGM3@0 for 194 with 4414 remaining (b = 0.01)
4655.01/591.61	c INFO: running hybridGM3@176 for 97 with 4409 remaining (b = 0.01)
4690.94/596.14	c INFO: running hybridGM3@292 for 194 with 4404 remaining (b = 0.02)
4802.29/610.14	c INFO: running TNM@0 for 97 with 4390 remaining (b = 0.01)
5462.60/693.07	c INFO: running adaptg2wsat2009++@0 for 194 with 4307 remaining (b = 0.01)
5476.02/694.70	c INFO: running TNM@0 for 97 with 4306 remaining (b = 0.01)
5600.23/710.32	c INFO: running hybridGM3@99 for 97 with 4290 remaining (b = 0.01)
5601.94/710.56	c INFO: running TNM@0 for 97 with 4290 remaining (b = 0.01)
5670.18/719.14	c INFO: running adaptg2wsat2009++@99 for 97 with 4281 remaining (b = 0.01)
6028.17/764.16	c INFO: running hybridGM3@171 for 97 with 4236 remaining (b = 0.01)
6184.14/783.74	c INFO: running hybridGM3@95 for 97 with 4216 remaining (b = 0.01)
6274.77/795.12	c INFO: running hybridGM3@195 for 291 with 4205 remaining (b = 0.02)
6400.46/810.98	c INFO: running TNM@0 for 97 with 4189 remaining (b = 0.01)
6406.92/811.73	c INFO: running TNM@0 for 97 with 4188 remaining (b = 0.01)
6469.64/819.60	c INFO: running adaptg2wsat2009++@296 for 291 with 4181 remaining (b = 0.02)
6787.04/859.55	c INFO: running hybridGM3@368 for 194 with 4141 remaining (b = 0.01)
6990.66/885.15	c INFO: running TNM@0 for 97 with 4115 remaining (b = 0.01)
7028.13/889.82	c INFO: running FH@0 for 97 with 4110 remaining (b = 0.01)
7199.81/911.41	c INFO: running TNM@0 for 97 with 4089 remaining (b = 0.01)
7205.34/912.16	c INFO: running TNM@0 for 97 with 4088 remaining (b = 0.00)
7789.23/985.58	c INFO: running TNM@0 for 97 with 4015 remaining (b = 0.00)
7826.40/990.25	c INFO: running hybridGM3@199 for 291 with 4010 remaining (b = 0.01)
7998.01/1011.84	c INFO: running TNM@0 for 97 with 3988 remaining (b = 0.00)
8003.56/1012.60	c INFO: running TNM@0 for 97 with 3988 remaining (b = 0.00)
8359.44/1057.31	c INFO: running hybridGM3@468 for 97 with 3943 remaining (b = 0.01)
8581.10/1085.21	c INFO: running hybridGM3@295 for 194 with 3915 remaining (b = 0.01)
8587.43/1086.02	c INFO: running TNM@0 for 97 with 3914 remaining (b = 0.00)
8593.24/1086.82	c INFO: running adaptg2wsat2009++@190 for 97 with 3913 remaining (b = 0.00)
8593.24/1086.88	c INFO: running hybridGM3@212 for 291 with 3913 remaining (b = 0.01)
8593.24/1086.88	Traceback (most recent call last):
8593.24/1086.88	  File "HOME/solve", line 33, in <module>
8593.24/1086.88	    plac.call(main)
8593.24/1086.88	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call
8593.24/1086.88	    cmd, result = parser_from(obj).consume(arglist)
8593.24/1086.88	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 192, in consume
8593.24/1086.88	    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
8593.24/1086.88	  File "HOME/borg/build/lib/borg/tools/solve.py", line 84, in main
8593.24/1086.88	    (_, answer, _) = solver(input_path, budget / cores, cores)
8593.24/1086.88	  File "HOME/borg/build/lib/borg/bilevel.py", line 202, in __call__
8593.24/1086.88	    solver.go(max_cost * borg.defaults.machine_speed)
8593.24/1086.88	  File "HOME/borg/build/lib/borg/solvers.py", line 179, in go
8593.24/1086.88	    self._process.start()
8593.24/1086.88	  File "/home/evaluation/evaluation/tools64/lib/python2.7/multiprocessing/process.py", line 94, in start
8593.24/1086.88	    assert self._popen is None, 'cannot start a process twice'
8593.24/1086.88	AssertionError: cannot start a process twice

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385352-1305565599/watcher-3385352-1305565599 -o /tmp/evaluation-result-3385352-1305565599/solver-3385352-1305565599 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385352-1305565599.cnf -seed 1461954144 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.17 7.10 3.96 1/430 29850
/proc/meminfo: memFree=21560708/32951124 swapFree=51873468/67111528
[pid=29850] ppid=29848 vsize=15904 CPUtime=0 cores=0-7
/proc/29850/stat : 29850 (python) S 29848 29850 29564 0 -1 4202496 957 0 0 0 0 0 0 0 21 0 1 0 648976211 16285696 833 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039653376 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/29850/statm: 3976 833 357 364 0 463 0

[startup+0.0281459 s]
/proc/loadavg: 8.17 7.10 3.96 1/430 29850
/proc/meminfo: memFree=21560708/32951124 swapFree=51873468/67111528
[pid=29850] ppid=29848 vsize=16392 CPUtime=0.01 cores=0-7
/proc/29850/stat : 29850 (python) S 29848 29850 29564 0 -1 4202496 1116 0 0 0 1 0 0 0 21 0 1 0 648976211 16785408 984 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039651173 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/29850/statm: 4098 984 409 364 0 585 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16392

[startup+0.101129 s]
/proc/loadavg: 8.17 7.10 3.96 1/430 29850
/proc/meminfo: memFree=21560708/32951124 swapFree=51873468/67111528
[pid=29850] ppid=29848 vsize=85620 CPUtime=0.07 cores=0-7
/proc/29850/stat : 29850 (python) R 29848 29850 29564 0 -1 4202496 2630 0 0 0 5 2 0 0 21 0 1 0 648976211 87674880 2348 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039651173 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/29850/statm: 21405 2348 713 364 0 1622 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 85620

[startup+0.301111 s]
/proc/loadavg: 8.17 7.10 3.96 1/430 29850
/proc/meminfo: memFree=21560708/32951124 swapFree=51873468/67111528
[pid=29850] ppid=29848 vsize=135748 CPUtime=0.19 cores=0-7
/proc/29850/stat : 29850 (python) S 29848 29850 29564 0 -1 4202496 5962 933 1 0 15 4 0 0 21 0 1 0 648976211 139005952 4151 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039651333 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/29850/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135748

[startup+0.701021 s]
/proc/loadavg: 8.17 7.10 3.96 1/430 29850
/proc/meminfo: memFree=21560708/32951124 swapFree=51873468/67111528
[pid=29850] ppid=29848 vsize=283272 CPUtime=0.45 cores=0-7
/proc/29850/stat : 29850 (python) R 29848 29850 29564 0 -1 4202496 10451 933 1 0 33 12 0 0 20 0 1 0 648976211 290070528 8135 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039341957 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/29850/statm: 70818 8135 1761 364 0 6742 0
Current children cumulated CPU time (s) 0.45
Current children cumulated vsize (KiB) 283272

[startup+1.50088 s]
/proc/loadavg: 8.17 7.10 3.96 2/431 29855
/proc/meminfo: memFree=21514696/32951124 swapFree=51873468/67111528
[pid=29850] ppid=29848 vsize=317988 CPUtime=1.25 cores=0-7
/proc/29850/stat : 29850 (python) R 29848 29850 29564 0 -1 4202496 23796 933 1 0 111 14 0 0 22 0 1 0 648976211 325619712 16882 33554432000 4194304 5683644 140736054684240 18446744073709551615 4623648 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/29850/statm: 79497 16882 1826 364 0 15421 0
Current children cumulated CPU time (s) 1.25
Current children cumulated vsize (KiB) 317988

[startup+3.10059 s]
/proc/loadavg: 8.17 7.10 3.96 2/431 29855
/proc/meminfo: memFree=21498824/32951124 swapFree=51873468/67111528
[pid=29850] ppid=29848 vsize=317988 CPUtime=2.85 cores=0-7
/proc/29850/stat : 29850 (python) R 29848 29850 29564 0 -1 4202496 23796 933 1 0 271 14 0 0 25 0 1 0 648976211 325619712 16882 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039315995 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/29850/statm: 79497 16882 1826 364 0 15421 0
Current children cumulated CPU time (s) 2.85
Current children cumulated vsize (KiB) 317988

[startup+6.30102 s]
/proc/loadavg: 7.59 6.99 3.95 2/431 29855
/proc/meminfo: memFree=21499576/32951124 swapFree=51873468/67111528
[pid=29850] ppid=29848 vsize=317988 CPUtime=6.05 cores=0-7
/proc/29850/stat : 29850 (python) R 29848 29850 29564 0 -1 4202496 23797 933 1 0 591 14 0 0 25 0 1 0 648976211 325619712 16883 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039304237 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/29850/statm: 79497 16883 1827 364 0 15421 0
Current children cumulated CPU time (s) 6.05
Current children cumulated vsize (KiB) 317988

[startup+12.7018 s]
/proc/loadavg: 7.07 6.89 3.93 9/455 29880
/proc/meminfo: memFree=21209972/32951124 swapFree=51873468/67111528
[pid=29850] ppid=29848 vsize=462456 CPUtime=8.55 cores=0-7
/proc/29850/stat : 29850 (python) S 29848 29850 29564 0 -1 4202496 29663 933 1 0 839 16 0 0 25 0 9 0 648976211 473554944 16360 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29850/statm: 115614 16360 1889 364 0 51511 0
[pid=29857] ppid=29850 vsize=314884 CPUtime=0 cores=0-7
/proc/29857/stat : 29857 (python) S 29850 29850 29564 0 -1 4202560 740 0 0 0 0 0 0 0 15 0 1 0 648977070 322441216 14627 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675407 0 0 16781312 514 11106562780646310146 0 0 17 1 0 0 0
/proc/29857/statm: 78721 14627 353 364 0 14639 0
[pid=29859] ppid=29857 vsize=276240 CPUtime=4.1 cores=0-7
/proc/29859/stat : 29859 (hybridGM3) R 29857 29859 29859 0 -1 4202496 8567 0 0 0 410 0 0 0 25 0 1 0 648977070 282869760 8303 33554432000 134512640 135204385 4289783328 18446744073709551615 134545968 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29859/statm: 69060 8303 64 169 0 68890 0
[pid=29860] ppid=29850 vsize=325140 CPUtime=0 cores=0-7
/proc/29860/stat : 29860 (python) S 29850 29850 29564 0 -1 4202560 738 0 0 0 0 0 0 0 15 0 1 0 648977073 332943360 14625 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 17240610608659792129 0 0 17 1 0 0 0
/proc/29860/statm: 81285 14625 347 364 0 17200 0
[pid=29862] ppid=29860 vsize=276240 CPUtime=4.07 cores=0-7
/proc/29862/stat : 29862 (hybridGM3) R 29860 29862 29862 0 -1 4202496 8565 0 0 0 407 0 0 0 25 0 1 0 648977073 282869760 8301 33554432000 134512640 135204385 4288033696 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/29862/statm: 69060 8301 64 169 0 68890 0
[pid=29863] ppid=29850 vsize=400932 CPUtime=0 cores=0-7
/proc/29863/stat : 29863 (python) S 29850 29850 29564 0 -1 4202560 704 0 0 0 0 0 0 0 15 0 1 0 648977076 410554368 14803 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 18193926772360184065 0 0 17 7 0 0 0
/proc/29863/statm: 100233 14803 348 364 0 36145 0
[pid=29865] ppid=29863 vsize=276240 CPUtime=4.04 cores=0-7
/proc/29865/stat : 29865 (hybridGM3) R 29863 29865 29865 0 -1 4202496 8556 0 0 0 403 1 0 0 25 0 1 0 648977076 282869760 8301 33554432000 134512640 135204385 4292278272 18446744073709551615 134546073 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/29865/statm: 69060 8301 64 169 0 68890 0
[pid=29866] ppid=29850 vsize=411188 CPUtime=0 cores=0-7
/proc/29866/stat : 29866 (python) S 29850 29850 29564 0 -1 4202560 718 0 0 0 0 0 0 0 15 0 1 0 648977079 421056512 14767 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 1401349564128002308 0 0 17 4 0 0 0
/proc/29866/statm: 102797 14767 348 364 0 38706 0
[pid=29868] ppid=29866 vsize=276244 CPUtime=4 cores=0-7
/proc/29868/stat : 29868 (hybridGM3) R 29866 29868 29868 0 -1 4202496 8554 0 0 0 399 1 0 0 25 0 1 0 648977079 282873856 8299 33554432000 134512640 135204385 4292827360 18446744073709551615 134537290 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/29868/statm: 69061 8299 64 169 0 68891 0
[pid=29869] ppid=29850 vsize=421444 CPUtime=0 cores=0-7
/proc/29869/stat : 29869 (python) S 29850 29850 29564 0 -1 4202560 725 0 0 0 0 0 0 0 15 0 1 0 648977082 431558656 14803 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 11600916402633933060 0 0 17 6 0 0 0
/proc/29869/statm: 105361 14803 348 364 0 41267 0
[pid=29871] ppid=29869 vsize=276240 CPUtime=3.98 cores=0-7
/proc/29871/stat : 29871 (hybridGM3) R 29869 29871 29871 0 -1 4202496 8555 0 0 0 397 1 0 0 25 0 1 0 648977083 282869760 8298 33554432000 134512640 135204385 4288488784 18446744073709551615 134546082 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/29871/statm: 69060 8298 64 169 0 68890 0
[pid=29872] ppid=29850 vsize=431700 CPUtime=0 cores=0-7
/proc/29872/stat : 29872 (python) S 29850 29850 29564 0 -1 4202560 734 0 0 0 0 0 0 0 15 0 1 0 648977086 442060800 14807 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 11600912004587421955 0 0 17 6 0 0 0
/proc/29872/statm: 107925 14807 348 364 0 43828 0
[pid=29874] ppid=29872 vsize=276240 CPUtime=3.88 cores=0-7
/proc/29874/stat : 29874 (hybridGM3) R 29872 29874 29874 0 -1 4202496 8557 0 0 0 387 1 0 0 25 0 1 0 648977087 282869760 8298 33554432000 134512640 135204385 4288302592 18446744073709551615 134536767 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/29874/statm: 69060 8298 64 169 0 68890 0
[pid=29875] ppid=29850 vsize=441956 CPUtime=0 cores=0-7
/proc/29875/stat : 29875 (python) S 29850 29850 29564 0 -1 4202560 741 0 0 0 0 0 0 0 15 0 1 0 648977089 452562944 14811 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 11465808413812818177 0 0 17 6 0 0 0
/proc/29875/statm: 110489 14811 348 364 0 46389 0
[pid=29877] ppid=29875 vsize=276240 CPUtime=3.86 cores=0-7
/proc/29877/stat : 29877 (hybridGM3) R 29875 29877 29877 0 -1 4202496 8560 0 0 0 384 2 0 0 25 0 1 0 648977090 282869760 8299 33554432000 134512640 135204385 4287570176 18446744073709551615 134537123 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/29877/statm: 69060 8299 64 169 0 68890 0
[pid=29878] ppid=29850 vsize=452212 CPUtime=0 cores=0-7
/proc/29878/stat : 29878 (python) S 29850 29850 29564 0 -1 4202560 777 0 0 0 0 0 0 0 15 0 1 0 648977093 463065088 14821 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 11166670882313568513 0 0 17 0 0 0 0
/proc/29878/statm: 113053 14821 354 364 0 48950 0
[pid=29880] ppid=29878 vsize=276240 CPUtime=3.87 cores=0-7
/proc/29880/stat : 29880 (hybridGM3) R 29878 29880 29880 0 -1 4202496 8562 0 0 0 387 0 0 0 25 0 1 0 648977093 282869760 8297 33554432000 134512640 135204385 4291761536 18446744073709551615 134547426 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/29880/statm: 69060 8297 64 169 0 68890 0
[pid=29850/tid=29858] ppid=29848 vsize=462456 CPUtime=0 cores=0-7
/proc/29850/task/29858/stat : 29858 (python) S 29848 29850 29564 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 9 0 648977070 473554944 16360 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29861] ppid=29848 vsize=462456 CPUtime=0 cores=0-7
/proc/29850/task/29861/stat : 29861 (python) S 29848 29850 29564 0 -1 4202560 25 933 0 0 0 0 0 0 18 0 9 0 648977073 473554944 16360 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 3 0 0 0
[pid=29850/tid=29864] ppid=29848 vsize=462456 CPUtime=0 cores=0-7
/proc/29850/task/29864/stat : 29864 (python) S 29848 29850 29564 0 -1 4202560 80 933 0 0 0 0 0 0 20 0 9 0 648977076 473554944 16360 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29867] ppid=29848 vsize=462456 CPUtime=0 cores=0-7
/proc/29850/task/29867/stat : 29867 (python) S 29848 29850 29564 0 -1 4202560 30 933 0 0 0 0 0 0 22 0 9 0 648977079 473554944 16360 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29870] ppid=29848 vsize=462456 CPUtime=0 cores=0-7
/proc/29850/task/29870/stat : 29870 (python) S 29848 29850 29564 0 -1 4202560 31 933 0 0 0 0 0 0 24 0 9 0 648977082 473554944 16360 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 6 0 0 0
[pid=29850/tid=29873] ppid=29848 vsize=462456 CPUtime=0 cores=0-7
/proc/29850/task/29873/stat : 29873 (python) S 29848 29850 29564 0 -1 4202560 32 933 0 0 0 0 0 0 25 0 9 0 648977086 473554944 16360 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 6 0 0 0
[pid=29850/tid=29876] ppid=29848 vsize=462456 CPUtime=0 cores=0-7
/proc/29850/task/29876/stat : 29876 (python) S 29848 29850 29564 0 -1 4202560 34 933 0 0 0 0 0 0 25 0 9 0 648977090 473554944 16360 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 6 0 0 0
[pid=29850/tid=29879] ppid=29848 vsize=462456 CPUtime=0 cores=0-7
/proc/29850/task/29879/stat : 29879 (python) S 29848 29850 29564 0 -1 4202560 35 933 0 0 0 0 0 0 25 0 9 0 648977093 473554944 16360 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40.35
Current children cumulated vsize (KiB) 5871836

[startup+25.5016 s]
/proc/loadavg: 7.27 6.95 3.99 10/455 29880
/proc/meminfo: memFree=21209872/32951124 swapFree=51873468/67111528
[pid=29850] ppid=29848 vsize=462456 CPUtime=8.55 cores=0-7
/proc/29850/stat : 29850 (python) S 29848 29850 29564 0 -1 4202496 29663 933 1 0 839 16 0 0 25 0 9 0 648976211 473554944 16360 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29850/statm: 115614 16360 1889 364 0 51511 0
[pid=29857] ppid=29850 vsize=314884 CPUtime=0.06 cores=0-7
/proc/29857/stat : 29857 (python) S 29850 29850 29564 0 -1 4202560 920 0 0 0 3 3 0 0 15 0 1 0 648977070 322441216 14650 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675407 0 0 16781312 514 12230439581798859010 0 0 17 1 0 0 0
/proc/29857/statm: 78721 14650 376 364 0 14639 0
[pid=29859] ppid=29857 vsize=276240 CPUtime=16.9 cores=0-7
/proc/29859/stat : 29859 (hybridGM3) R 29857 29859 29859 0 -1 4202496 8567 0 0 0 1690 0 0 0 25 0 1 0 648977070 282869760 8303 33554432000 134512640 135204385 4289783328 18446744073709551615 134536663 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29859/statm: 69060 8303 64 169 0 68890 0
[pid=29860] ppid=29850 vsize=325140 CPUtime=0.05 cores=0-7
/proc/29860/stat : 29860 (python) S 29850 29850 29564 0 -1 4202560 912 0 0 0 3 2 0 0 15 0 1 0 648977073 332943360 14648 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 12230439581798859009 0 0 17 1 0 0 0
/proc/29860/statm: 81285 14648 370 364 0 17200 0
[pid=29862] ppid=29860 vsize=276240 CPUtime=16.86 cores=0-7
/proc/29862/stat : 29862 (hybridGM3) R 29860 29862 29862 0 -1 4202496 8565 0 0 0 1686 0 0 0 25 0 1 0 648977073 282869760 8301 33554432000 134512640 135204385 4288033696 18446744073709551615 134547676 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/29862/statm: 69060 8301 64 169 0 68890 0
[pid=29863] ppid=29850 vsize=400932 CPUtime=0.05 cores=0-7
/proc/29863/stat : 29863 (python) S 29850 29850 29564 0 -1 4202560 782 0 0 0 2 3 0 0 15 0 1 0 648977076 410554368 14826 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 12230439581798859009 0 0 17 7 0 0 0
/proc/29863/statm: 100233 14826 371 364 0 36145 0
[pid=29865] ppid=29863 vsize=276240 CPUtime=16.84 cores=0-7
/proc/29865/stat : 29865 (hybridGM3) R 29863 29865 29865 0 -1 4202496 8556 0 0 0 1683 1 0 0 25 0 1 0 648977076 282869760 8301 33554432000 134512640 135204385 4292278272 18446744073709551615 134537440 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/29865/statm: 69060 8301 64 169 0 68890 0
[pid=29866] ppid=29850 vsize=411188 CPUtime=0.05 cores=0-7
/proc/29866/stat : 29866 (python) S 29850 29850 29564 0 -1 4202560 798 0 0 0 2 3 0 0 16 0 1 0 648977079 421056512 14790 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 12230439581798859012 0 0 17 4 0 0 0
/proc/29866/statm: 102797 14790 371 364 0 38706 0
[pid=29868] ppid=29866 vsize=276244 CPUtime=16.74 cores=0-7
/proc/29868/stat : 29868 (hybridGM3) R 29866 29868 29868 0 -1 4202496 8554 0 0 0 1673 1 0 0 25 0 1 0 648977079 282873856 8299 33554432000 134512640 135204385 4292827360 18446744073709551615 134537510 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/29868/statm: 69061 8299 64 169 0 68891 0
[pid=29869] ppid=29850 vsize=421444 CPUtime=0.05 cores=0-7
/proc/29869/stat : 29869 (python) S 29850 29850 29564 0 -1 4202560 802 0 0 0 2 3 0 0 16 0 1 0 648977082 431558656 14826 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 12230439581798859012 0 0 17 6 0 0 0
/proc/29869/statm: 105361 14826 371 364 0 41267 0
[pid=29871] ppid=29869 vsize=276240 CPUtime=16.66 cores=0-7
/proc/29871/stat : 29871 (hybridGM3) R 29869 29871 29871 0 -1 4202496 8555 0 0 0 1665 1 0 0 25 0 1 0 648977083 282869760 8298 33554432000 134512640 135204385 4288488784 18446744073709551615 134537272 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/29871/statm: 69060 8298 64 169 0 68890 0

################
# More data... #
################

/proc/30001/statm: 174589 15246 406 364 0 110414 0
[pid=30003] ppid=30001 vsize=467024 CPUtime=97.99 cores=0-7
/proc/30003/stat : 30003 (TNM) T 30001 30003 30003 0 -1 4202496 6848 0 0 0 9798 1 0 0 25 0 1 0 649057310 478232576 6559 33554432000 134512640 135008748 4292099744 18446744073709551615 134527750 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/30003/statm: 116756 6559 46 122 0 115656 0
[pid=30001/tid=30017] ppid=29850 vsize=698356 CPUtime=0 cores=0-7
/proc/30001/task/30017/stat : 30017 (python) S 29850 29850 29564 0 -1 4202560 90 0 0 0 0 0 0 0 19 0 2 0 649067346 715116544 15246 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 514 18446604451164628288 0 0 -1 3 0 0 0
[pid=30004] ppid=29850 vsize=708612 CPUtime=0.37 cores=0-7
/proc/30004/stat : 30004 (python) S 29850 29850 29564 0 -1 4202560 1250 0 0 0 18 19 0 0 19 0 2 0 649057385 725618688 15299 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30004/statm: 177153 15299 406 364 0 112975 0
[pid=30006] ppid=30004 vsize=467024 CPUtime=100.21 cores=0-7
/proc/30006/stat : 30006 (TNM) T 30004 30006 30006 0 -1 4202496 6851 0 0 0 10020 1 0 0 25 0 1 0 649057386 478232576 6559 33554432000 134512640 135008748 4293504320 18446744073709551615 134528988 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30006/statm: 116756 6559 46 122 0 115656 0
[pid=30004/tid=30021] ppid=29850 vsize=708612 CPUtime=0 cores=0-7
/proc/30004/task/30021/stat : 30021 (python) S 29850 29850 29564 0 -1 4202560 88 0 0 0 0 0 0 0 19 0 2 0 649067422 725618688 15299 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 514 18446604451164629184 0 0 -1 0 0 0 0
[pid=30009] ppid=29850 vsize=718868 CPUtime=0.37 cores=0-7
/proc/30009/stat : 30009 (python) S 29850 29850 29564 0 -1 4202560 1254 0 0 0 17 20 0 0 18 0 2 0 649064727 736120832 15219 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30009/statm: 179717 15219 401 364 0 115536 0
[pid=30011] ppid=30009 vsize=467024 CPUtime=99.99 cores=0-7
/proc/30011/stat : 30011 (TNM) T 30009 30011 30011 0 -1 4202496 6848 0 0 0 9998 1 0 0 25 0 1 0 649064727 478232576 6560 33554432000 134512640 135008748 4292527488 18446744073709551615 134524772 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/30011/statm: 116756 6560 46 122 0 115656 0
[pid=30009/tid=30027] ppid=29850 vsize=718868 CPUtime=0 cores=0-7
/proc/30009/task/30027/stat : 30027 (python) S 29850 29850 29564 0 -1 4202560 90 0 0 0 0 0 0 0 19 0 2 0 649074763 736120832 15219 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 514 18446604442071685120 0 0 -1 6 0 0 0
[pid=30013] ppid=29850 vsize=729124 CPUtime=0.37 cores=0-7
/proc/30013/stat : 30013 (python) S 29850 29850 29564 0 -1 4202560 1238 0 0 0 20 17 0 0 19 0 2 0 649065194 746622976 15288 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30013/statm: 182281 15288 400 364 0 118097 0
[pid=30015] ppid=30013 vsize=262384 CPUtime=99.77 cores=0-7
/proc/30015/stat : 30015 (FH) T 30013 30015 30015 0 -1 4202496 7017 0 0 0 9976 1 0 0 25 0 1 0 649065194 268681216 6725 33554432000 134512640 134936988 4291582624 18446744073709551615 134553498 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/30015/statm: 65596 6725 41 104 0 65100 0
[pid=30013/tid=30031] ppid=29850 vsize=729124 CPUtime=0 cores=0-7
/proc/30013/task/30031/stat : 30031 (python) S 29850 29850 29564 0 -1 4202560 90 0 0 0 0 0 0 0 19 0 2 0 649075231 746622976 15288 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 514 18446604450366315392 0 0 -1 3 0 0 0
[pid=30018] ppid=29850 vsize=739380 CPUtime=0.37 cores=0-7
/proc/30018/stat : 30018 (python) S 29850 29850 29564 0 -1 4202560 1302 0 0 0 20 17 0 0 19 0 2 0 649067353 757125120 15243 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30018/statm: 184845 15243 407 364 0 120658 0
[pid=30020] ppid=30018 vsize=467024 CPUtime=100.28 cores=0-7
/proc/30020/stat : 30020 (TNM) T 30018 30020 30020 0 -1 4202496 6857 0 0 0 10027 1 0 0 25 0 1 0 649067353 478232576 6560 33554432000 134512640 135008748 4294152128 18446744073709551615 134529155 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30020/statm: 116756 6560 46 122 0 115656 0
[pid=30018/tid=30032] ppid=29850 vsize=739380 CPUtime=0 cores=0-7
/proc/30018/task/30032/stat : 30032 (python) S 29850 29850 29564 0 -1 4202560 96 0 0 0 0 0 0 0 19 0 2 0 649077389 757125120 15243 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 514 18446604450022202112 0 0 -1 6 0 0 0
[pid=30022] ppid=29850 vsize=749636 CPUtime=0.38 cores=0-7
/proc/30022/stat : 30022 (python) S 29850 29850 29564 0 -1 4202560 1284 0 0 0 17 21 0 0 19 0 2 0 649067428 767627264 15323 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/30022/statm: 187409 15323 407 364 0 123219 0
[pid=30024] ppid=30022 vsize=467024 CPUtime=99.13 cores=0-7
/proc/30024/stat : 30024 (TNM) T 30022 30024 30024 0 -1 4202496 6853 0 0 0 9911 2 0 0 25 0 1 0 649067429 478232576 6559 33554432000 134512640 135008748 4288375696 18446744073709551615 134528467 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30024/statm: 116756 6559 46 122 0 115656 0
[pid=30022/tid=30036] ppid=29850 vsize=749636 CPUtime=0 cores=0-7
/proc/30022/task/30036/stat : 30036 (python) S 29850 29850 29564 0 -1 4202560 92 0 0 0 0 0 0 0 19 0 2 0 649077465 767627264 15323 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 514 18446604440361208512 0 0 -1 4 0 0 0
[pid=30028] ppid=29850 vsize=759892 CPUtime=0.37 cores=0-7
/proc/30028/stat : 30028 (python) S 29850 29850 29564 0 -1 4202560 1305 0 0 0 21 16 0 0 19 0 2 0 649074770 778129408 15263 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30028/statm: 189973 15263 407 364 0 125780 0
[pid=30030] ppid=30028 vsize=467024 CPUtime=99.92 cores=0-7
/proc/30030/stat : 30030 (TNM) T 30028 30030 30030 0 -1 4202496 6854 0 0 0 9990 2 0 0 25 0 1 0 649074770 478232576 6560 33554432000 134512640 135008748 4287287760 18446744073709551615 134529448 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/30030/statm: 116756 6560 46 122 0 115656 0
[pid=30028/tid=30042] ppid=29850 vsize=759892 CPUtime=0 cores=0-7
/proc/30028/task/30042/stat : 30042 (python) S 29850 29850 29564 0 -1 4202560 92 0 0 0 0 0 0 0 19 0 2 0 649084807 778129408 15263 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 514 18446604447449194752 0 0 -1 0 0 0 0
[pid=30033] ppid=29850 vsize=770148 CPUtime=0.26 cores=0-7
/proc/30033/stat : 30033 (python) S 29850 29850 29564 0 -1 4202560 1089 0 0 0 14 12 0 0 18 0 1 0 649077396 788631552 15243 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 12230439581798859010 0 0 17 0 0 0 0
/proc/30033/statm: 192537 15243 379 364 0 128341 0
[pid=30035] ppid=30033 vsize=467024 CPUtime=74.72 cores=0-7
/proc/30035/stat : 30035 (TNM) R 30033 30035 30035 0 -1 4202496 6858 0 0 0 7470 2 0 0 25 0 1 0 649077396 478232576 6559 33554432000 134512640 135008748 4294019152 18446744073709551615 134527840 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/30035/statm: 116756 6559 46 122 0 115656 0
[pid=30037] ppid=29850 vsize=780404 CPUtime=0.25 cores=0-7
/proc/30037/stat : 30037 (python) S 29850 29850 29564 0 -1 4202560 1129 0 0 0 12 13 0 0 18 0 1 0 649077472 799133696 15331 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 12230439581798859011 0 0 17 0 0 0 0
/proc/30037/statm: 195101 15331 378 364 0 130902 0
[pid=30039] ppid=30037 vsize=467024 CPUtime=73.41 cores=0-7
/proc/30039/stat : 30039 (TNM) R 30037 30039 30039 0 -1 4202496 6856 0 0 0 7339 2 0 0 25 0 1 0 649077472 478232576 6559 33554432000 134512640 135008748 4288467664 18446744073709551615 134527693 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30039/statm: 116756 6559 46 122 0 115656 0
[pid=30043] ppid=29850 vsize=790660 CPUtime=0 cores=0-7
/proc/30043/stat : 30043 (python) S 29850 29850 29564 0 -1 4202560 986 0 0 0 0 0 0 0 19 0 1 0 649084813 809635840 15263 33554432000 4194304 5683644 140736054684240 18446744073709551615 212039675462 0 0 16781312 514 16374318316396740607 0 0 17 1 0 0 0
/proc/30043/statm: 197665 15263 351 364 0 133463 0
[pid=30045] ppid=30043 vsize=467024 CPUtime=0.71 cores=0-7
/proc/30045/stat : 30045 (TNM) R 30043 30045 30045 0 -1 4202496 6854 0 0 0 70 1 0 0 22 0 1 0 649084814 478232576 6559 33554432000 134512640 135008748 4290927840 18446744073709551615 134528182 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/30045/statm: 116756 6559 46 122 0 115656 0
[pid=29850/tid=29858] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29858/stat : 29858 (python) S 29848 29850 29564 0 -1 4202560 28 98722 0 0 0 0 222377 479 25 0 42 0 648977070 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29861] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29861/stat : 29861 (python) S 29848 29850 29564 0 -1 4202560 31 98722 0 0 0 0 222377 479 15 0 42 0 648977073 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 3 0 0 0
[pid=29850/tid=29864] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29864/stat : 29864 (python) S 29848 29850 29564 0 -1 4202560 95 98722 0 0 0 0 222377 479 18 0 42 0 648977076 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29867] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29867/stat : 29867 (python) S 29848 29850 29564 0 -1 4202560 35 98722 0 0 0 0 222377 479 18 0 42 0 648977079 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 7 0 0 0
[pid=29850/tid=29870] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29870/stat : 29870 (python) S 29848 29850 29564 0 -1 4202560 50 98722 0 0 0 0 222377 479 18 0 42 0 648977082 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29873] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29873/stat : 29873 (python) S 29848 29850 29564 0 -1 4202560 50 98722 0 0 0 0 222377 479 16 0 42 0 648977086 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 2 0 0 0
[pid=29850/tid=29876] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29876/stat : 29876 (python) S 29848 29850 29564 0 -1 4202560 39 98722 0 0 0 0 222377 479 18 0 42 0 648977090 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 6 0 0 0
[pid=29850/tid=29879] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29879/stat : 29879 (python) S 29848 29850 29564 0 -1 4202560 42 98722 0 0 0 0 222377 479 18 0 42 0 648977093 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 6 0 0 0
[pid=29850/tid=29885] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29885/stat : 29885 (python) S 29848 29850 29564 0 -1 4202560 40 98722 0 0 0 0 222377 479 20 0 42 0 648985891 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 3 0 0 0
[pid=29850/tid=29893] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29893/stat : 29893 (python) S 29848 29850 29564 0 -1 4202560 47 98722 0 0 0 0 222377 479 18 0 42 0 648988386 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29896] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29896/stat : 29896 (python) S 29848 29850 29564 0 -1 4202560 40 98722 0 0 0 0 222377 479 18 0 42 0 648989261 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29902] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29902/stat : 29902 (python) S 29848 29850 29564 0 -1 4202560 40 98722 0 0 0 0 222377 479 18 0 42 0 648995928 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 1 0 0 0
[pid=29850/tid=29908] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29908/stat : 29908 (python) S 29848 29850 29564 0 -1 4202560 51 98722 0 0 0 0 222377 479 18 0 42 0 648999019 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29913] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29913/stat : 29913 (python) S 29848 29850 29564 0 -1 4202560 47 98722 0 0 0 0 222377 479 18 0 42 0 649005499 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29917] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29917/stat : 29917 (python) S 29848 29850 29564 0 -1 4202560 47 98722 0 0 0 0 222377 479 18 0 42 0 649005968 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29921] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29921/stat : 29921 (python) S 29848 29850 29564 0 -1 4202560 37 98722 0 0 0 0 222377 479 20 0 42 0 649006934 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29924] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29924/stat : 29924 (python) S 29848 29850 29564 0 -1 4202560 34 98722 0 0 0 0 222377 479 20 0 42 0 649007029 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29932] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29932/stat : 29932 (python) S 29848 29850 29564 0 -1 4202560 37 98722 0 0 0 0 222377 479 20 0 42 0 649016972 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 1 0 0 0
[pid=29850/tid=29936] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29936/stat : 29936 (python) S 29848 29850 29564 0 -1 4202560 33 98722 0 0 0 0 222377 479 19 0 42 0 649017479 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29939] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29939/stat : 29939 (python) S 29848 29850 29564 0 -1 4202560 33 98722 0 0 0 0 222377 479 20 0 42 0 649017637 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29943] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29943/stat : 29943 (python) S 29848 29850 29564 0 -1 4202560 34 98722 0 0 0 0 222377 479 20 0 42 0 649018511 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29948] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29948/stat : 29948 (python) S 29848 29850 29564 0 -1 4202560 34 98722 0 0 0 0 222377 479 20 0 42 0 649022860 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29954] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29954/stat : 29954 (python) S 29848 29850 29564 0 -1 4202560 38 98722 0 0 0 0 222377 479 20 0 42 0 649026558 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 1 0 0 0
[pid=29850/tid=29959] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29959/stat : 29959 (python) S 29848 29850 29564 0 -1 4202560 36 98722 0 0 0 0 222377 479 18 0 42 0 649027519 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29963] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29963/stat : 29963 (python) S 29848 29850 29564 0 -1 4202560 43 98722 0 0 0 0 222377 479 18 0 42 0 649028552 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29968] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29968/stat : 29968 (python) S 29848 29850 29564 0 -1 4202560 45 98722 0 0 0 0 222377 479 18 0 42 0 649032901 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29973] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29973/stat : 29973 (python) S 29848 29850 29564 0 -1 4202560 38 98722 0 0 0 0 222377 479 18 0 42 0 649034867 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29977] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29977/stat : 29977 (python) S 29848 29850 29564 0 -1 4202560 34 98722 0 0 0 0 222377 479 19 0 42 0 649037226 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29982] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29982/stat : 29982 (python) S 29848 29850 29564 0 -1 4202560 34 98722 0 0 0 0 222377 479 20 0 42 0 649045520 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 1 0 0 0
[pid=29850/tid=29986] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29986/stat : 29986 (python) S 29848 29850 29564 0 -1 4202560 32 98722 0 0 0 0 222377 479 20 0 42 0 649045682 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=29991] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/29991/stat : 29991 (python) S 29848 29850 29564 0 -1 4202560 34 98722 0 0 0 0 222377 479 20 0 42 0 649047268 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 1 0 0 0
[pid=29850/tid=30002] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/30002/stat : 30002 (python) S 29848 29850 29564 0 -1 4202560 35 98722 0 0 0 0 222377 479 20 0 42 0 649057310 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 1 0 0 0
[pid=29850/tid=30005] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/30005/stat : 30005 (python) S 29848 29850 29564 0 -1 4202560 33 98722 0 0 0 0 222377 479 20 0 42 0 649057386 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=30010] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/30010/stat : 30010 (python) S 29848 29850 29564 0 -1 4202560 35 98722 0 0 0 0 222377 479 20 0 42 0 649064727 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 1 0 0 0
[pid=29850/tid=30014] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/30014/stat : 30014 (python) S 29848 29850 29564 0 -1 4202560 33 98722 0 0 0 0 222377 479 20 0 42 0 649065194 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=30019] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/30019/stat : 30019 (python) S 29848 29850 29564 0 -1 4202560 33 98722 0 0 0 0 222377 479 20 0 42 0 649067353 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=30023] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/30023/stat : 30023 (python) S 29848 29850 29564 0 -1 4202560 33 98722 0 0 0 0 222377 479 19 0 42 0 649067428 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=30029] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/30029/stat : 30029 (python) S 29848 29850 29564 0 -1 4202560 32 98722 0 0 0 0 222377 479 20 0 42 0 649074770 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 1 0 0 0
[pid=29850/tid=30034] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/30034/stat : 30034 (python) S 29848 29850 29564 0 -1 4202560 35 98722 0 0 0 0 222377 479 20 0 42 0 649077396 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=30038] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/30038/stat : 30038 (python) S 29848 29850 29564 0 -1 4202560 33 98722 0 0 0 0 222377 479 20 0 42 0 649077472 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
[pid=29850/tid=30044] ppid=29848 vsize=800904 CPUtime=2228.56 cores=0-7
/proc/29850/task/30044/stat : 30044 (python) S 29848 29850 29564 0 -1 4202560 39 98722 0 0 0 0 222377 479 19 0 42 0 649084814 820125696 16863 33554432000 4194304 5683644 140736054684240 18446744073709551615 212051478961 0 0 16781312 2 18446604450366314496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 8593.24
Current children cumulated vsize (KiB) 32782268

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 29850 and gives
#  childrusage.ru_utime.tv_sec=5120
#  childrusage.ru_utime.tv_usec=915502
#  childrusage.ru_stime.tv_sec=17
#  childrusage.ru_stime.tv_usec=6414
# CPU time returned by wait4() is 5137.92
# while last known CPU time is 8593.24
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1086.91
CPU time (s): 8593.24
CPU user time (s): 8576.12
CPU system time (s): 17.12
CPU usage (%): 790.61
Max. virtual memory (cumulated for all children) (KiB): 32782268

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5120.92
system time used= 17.0064
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 298246
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 15296
involuntary context switches= 34015

runsolver used 10.6404 second user time and 27.5348 second system time

The end

Launcher Data 

Begin job on node146 at 2011-05-16 19:06:39
IDJOB=3385352
IDBENCH=82617
IDSOLVER=1870
FILE ID=node146/3385352-1305565599
RUNJOBID= node146-1305564961-29578
PBS_JOBID= 13350104
Free space on /tmp= 73220 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v40000-c168000-S64788608-008-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385352-1305565599/watcher-3385352-1305565599 -o /tmp/evaluation-result-3385352-1305565599/solver-3385352-1305565599 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385352-1305565599.cnf -seed 1461954144 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= c0e1e71ade06eb64bd0f5dbaf6b892a1
RANDOM SEED=1461954144

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21560864 kB
Buffers:        209632 kB
Cached:        1433032 kB
SwapCached:      56140 kB
Active:        8282748 kB
Inactive:      2838836 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21560864 kB
SwapTotal:    67111528 kB
SwapFree:     51873468 kB
Dirty:           35064 kB
Writeback:          28 kB
AnonPages:     9430116 kB
Mapped:          22724 kB
Slab:           115204 kB
PageTables:      81360 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 118927948 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72864 MiB
End job on node146 at 2011-05-16 19:24:48