Trace number 3385344

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 7086.74 896.674

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v35000-c147000-S648642831-094-UNKNOWN.cnf
MD5SUM4a57833314362e0939d4095b3206ad82
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark144.491
Satisfiable
(Un)Satisfiability was proved
Number of variables35000
Number of clauses147000
Sum of the clauses size441000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3147000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.36/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.36/0.67	c INFO: solving HOME/instance-3385344-1305565399.cnf
0.36/0.67	c INFO: computing task features
0.85/1.17	c INFO: computed variable-clause graph statistics
0.85/1.19	c INFO: computed clause balance statistics
3.44/3.78	c INFO: computed variable balance statistics
5.53/5.88	c INFO: computed small-clause counts
5.53/5.89	c INFO: computed Horn-clause counts
7.13/7.45	c INFO: computed variable graph statistics
7.13/7.45	c INFO: collected features for HOME/instance-3385344-1305565399.cnf in 6.75 s
7.13/7.48	c INFO: running hybridGM3@0 for 97 with 4993 remaining (b = 0.36)
7.24/7.51	c INFO: running hybridGM3@0 for 97 with 4993 remaining (b = 0.22)
7.24/7.54	c INFO: running hybridGM3@0 for 97 with 4993 remaining (b = 0.14)
7.24/7.57	c INFO: running hybridGM3@0 for 97 with 4993 remaining (b = 0.10)
7.24/7.60	c INFO: running hybridGM3@0 for 97 with 4993 remaining (b = 0.07)
7.61/7.64	c INFO: running hybridGM3@0 for 97 with 4993 remaining (b = 0.05)
7.61/7.67	c INFO: running hybridGM3@0 for 194 with 4993 remaining (b = 0.08)
7.61/7.70	c INFO: running hybridGM3@0 for 194 with 4992 remaining (b = 0.06)
800.24/106.95	c INFO: running hybridGM3@0 for 97 with 4893 remaining (b = 0.03)
807.48/107.88	c INFO: running hybridGM3@0 for 97 with 4892 remaining (b = 0.03)
808.18/107.91	c INFO: running hybridGM3@0 for 97 with 4892 remaining (b = 0.03)
808.18/107.97	c INFO: running hybridGM3@0 for 97 with 4892 remaining (b = 0.03)
808.93/108.00	c INFO: running hybridGM3@0 for 97 with 4892 remaining (b = 0.03)
808.93/108.06	c INFO: running hybridGM3@0 for 97 with 4892 remaining (b = 0.03)
1560.44/202.24	c INFO: running hybridGM3@100 for 97 with 4798 remaining (b = 0.03)
1567.53/203.16	c INFO: running hybridGM3@100 for 97 with 4797 remaining (b = 0.03)
1567.53/203.20	c INFO: running hybridGM3@100 for 97 with 4797 remaining (b = 0.03)
1568.31/203.26	c INFO: running TNM@0 for 97 with 4797 remaining (b = 0.03)
1568.31/203.30	c INFO: running hybridGM3@99 for 97 with 4797 remaining (b = 0.03)
1577.02/204.35	c INFO: running hybridGM3@98 for 97 with 4796 remaining (b = 0.03)
1616.95/209.35	c INFO: running hybridGM3@0 for 97 with 4791 remaining (b = 0.03)
1624.90/210.39	c INFO: running hybridGM3@95 for 97 with 4790 remaining (b = 0.03)
2368.57/303.54	c INFO: running hybridGM3@99 for 97 with 4697 remaining (b = 0.03)
2368.57/303.60	c INFO: running hybridGM3@0 for 194 with 4697 remaining (b = 0.04)
2369.26/303.67	c INFO: running hybridGM3@0 for 194 with 4697 remaining (b = 0.03)
2374.02/304.47	c INFO: running TNM@0 for 97 with 4696 remaining (b = 0.02)
2374.63/304.51	c INFO: running TNM@0 for 97 with 4696 remaining (b = 0.01)
2374.63/304.61	c INFO: running hybridGM3@295 for 97 with 4696 remaining (b = 0.02)
2377.07/304.69	c INFO: running TNM@0 for 97 with 4696 remaining (b = 0.01)
2392.94/306.67	c INFO: running TNM@0 for 97 with 4694 remaining (b = 0.01)
3136.79/399.91	c INFO: running hybridGM3@388 for 97 with 4600 remaining (b = 0.01)
3143.97/400.86	c INFO: running TNM@0 for 97 with 4599 remaining (b = 0.01)
3175.94/404.82	c INFO: running TNM@0 for 97 with 4595 remaining (b = 0.01)
3175.94/404.89	c INFO: running TNM@0 for 97 with 4595 remaining (b = 0.01)
3177.17/405.03	c INFO: running TNM@0 for 2037 with 4595 remaining (b = 0.19)
3192.55/407.01	c INFO: running hybridGM3@0 for 194 with 4593 remaining (b = 0.02)
3895.93/495.23	c INFO: running hybridGM3@0 for 194 with 4505 remaining (b = 0.02)
3943.72/501.24	c INFO: running hybridGM3@0 for 194 with 4499 remaining (b = 0.02)
3953.26/502.50	c INFO: running hybridGM3@101 for 97 with 4498 remaining (b = 0.01)
3974.75/505.20	c INFO: running adaptg2wsat2009++@0 for 194 with 4495 remaining (b = 0.02)
3975.52/505.27	c INFO: running hybridGM3@0 for 194 with 4495 remaining (b = 0.01)
3985.00/506.48	c INFO: running hybridGM3@99 for 97 with 4494 remaining (b = 0.01)
4696.37/595.84	c INFO: running TNM@0 for 97 with 4404 remaining (b = 0.01)
4751.31/602.81	c INFO: running hybridGM3@95 for 97 with 4397 remaining (b = 0.01)
4791.85/607.85	c INFO: running hybridGM3@192 for 97 with 4392 remaining (b = 0.01)
5445.94/690.06	c INFO: running hybridGM3@100 for 97 with 4310 remaining (b = 0.01)
5495.30/696.22	c INFO: running adaptg2wsat2009++@0 for 194 with 4304 remaining (b = 0.01)
5539.87/701.84	c INFO: running hybridGM3@0 for 194 with 4298 remaining (b = 0.01)
5541.88/702.12	c INFO: running TNM@0 for 97 with 4298 remaining (b = 0.01)
5550.84/703.21	c INFO: running hybridGM3@386 for 194 with 4297 remaining (b = 0.01)
5557.29/704.07	c INFO: running TNM@0 for 97 with 4296 remaining (b = 0.01)
5557.94/704.17	c INFO: running TNM@0 for 97 with 4296 remaining (b = 0.01)
6252.22/791.44	c INFO: running TNM@0 for 97 with 4209 remaining (b = 0.01)
6332.44/801.52	c INFO: running adaptg2wsat2009++@100 for 97 with 4199 remaining (b = 0.01)
6355.48/804.49	c INFO: running hybridGM3@195 for 291 with 4196 remaining (b = 0.02)
6388.87/808.65	c INFO: running TNM@0 for 97 with 4192 remaining (b = 0.01)
7050.81/891.91	c INFO: running TNM@0 for 97 with 4108 remaining (b = 0.01)
7082.59/895.99	c INFO: running TNM@0 for 97 with 4104 remaining (b = 0.01)
7086.74/896.60	c INFO: running hybridGM3@198 for 291 with 4104 remaining (b = 0.02)
7086.74/896.60	Traceback (most recent call last):
7086.74/896.60	  File "HOME/solve", line 33, in <module>
7086.74/896.60	    plac.call(main)
7086.74/896.60	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call
7086.74/896.60	    cmd, result = parser_from(obj).consume(arglist)
7086.74/896.60	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 192, in consume
7086.74/896.60	    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
7086.74/896.60	  File "HOME/borg/build/lib/borg/tools/solve.py", line 84, in main
7086.74/896.60	    (_, answer, _) = solver(input_path, budget / cores, cores)
7086.74/896.60	  File "HOME/borg/build/lib/borg/bilevel.py", line 202, in __call__
7086.74/896.60	    solver.go(max_cost * borg.defaults.machine_speed)
7086.74/896.60	  File "HOME/borg/build/lib/borg/solvers.py", line 179, in go
7086.74/896.60	    self._process.start()
7086.74/896.60	  File "/home/evaluation/evaluation/tools64/lib/python2.7/multiprocessing/process.py", line 94, in start
7086.74/896.60	    assert self._popen is None, 'cannot start a process twice'
7086.74/896.60	AssertionError: cannot start a process twice

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385344-1305565399/watcher-3385344-1305565399 -o /tmp/evaluation-result-3385344-1305565399/solver-3385344-1305565399 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385344-1305565399.cnf -seed 1889413889 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.03 6.16 2.99 1/363 8045
/proc/meminfo: memFree=22308600/32951124 swapFree=67057896/67111528
[pid=8045] ppid=8043 vsize=15760 CPUtime=0 cores=0-7
/proc/8045/stat : 8045 (python) R 8043 8045 7833 0 -1 4202496 844 0 0 0 0 0 0 0 17 0 1 0 648955302 16138240 727 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261301710 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/8045/statm: 3940 727 334 364 0 427 0

[startup+0.0327851 s]
/proc/loadavg: 8.03 6.16 2.99 1/363 8045
/proc/meminfo: memFree=22308600/32951124 swapFree=67057896/67111528
[pid=8045] ppid=8043 vsize=16536 CPUtime=0.01 cores=0-7
/proc/8045/stat : 8045 (python) S 8043 8045 7833 0 -1 4202496 1185 0 0 0 1 0 0 0 17 0 1 0 648955302 16932864 1046 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261614592 0 2147483391 16781312 2 18446744071699207358 0 0 17 4 0 0 0
/proc/8045/statm: 4134 1046 417 364 0 621 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16536

[startup+0.100777 s]
/proc/loadavg: 8.03 6.16 2.99 1/363 8045
/proc/meminfo: memFree=22308600/32951124 swapFree=67057896/67111528
[pid=8045] ppid=8043 vsize=87704 CPUtime=0.05 cores=0-7
/proc/8045/stat : 8045 (python) S 8043 8045 7833 0 -1 4202496 2637 0 0 0 4 1 0 0 17 0 1 0 648955302 89808896 2354 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261614592 0 2147483391 16781312 2 18446744071699207358 0 0 17 4 0 0 0
/proc/8045/statm: 21927 2354 717 364 0 1623 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 87704

[startup+0.30074 s]
/proc/loadavg: 8.03 6.16 2.99 1/363 8045
/proc/meminfo: memFree=22308600/32951124 swapFree=67057896/67111528
[pid=8045] ppid=8043 vsize=135748 CPUtime=0.19 cores=0-7
/proc/8045/stat : 8045 (python) R 8043 8045 7833 0 -1 4202496 5963 933 1 0 14 5 0 0 17 0 1 0 648955302 139005952 4151 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261612549 0 2147483391 16781312 2 0 0 0 17 4 0 0 0
/proc/8045/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135748

[startup+0.700668 s]
/proc/loadavg: 8.03 6.16 2.99 1/363 8045
/proc/meminfo: memFree=22308600/32951124 swapFree=67057896/67111528
[pid=8045] ppid=8043 vsize=282832 CPUtime=0.44 cores=0-7
/proc/8045/stat : 8045 (python) R 8043 8045 7833 0 -1 4202496 10350 933 1 0 32 12 0 0 16 0 1 0 648955302 289619968 8033 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261302992 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/8045/statm: 70708 8033 1761 364 0 6632 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 282832

[startup+1.50053 s]
/proc/loadavg: 7.46 6.07 2.98 2/364 8050
/proc/meminfo: memFree=22263208/32951124 swapFree=67057896/67111528
[pid=8045] ppid=8043 vsize=312972 CPUtime=1.24 cores=0-7
/proc/8045/stat : 8045 (python) R 8043 8045 7833 0 -1 4202496 21966 933 1 0 110 14 0 0 17 0 1 0 648955302 320483328 15628 33554432000 4194304 5683644 140736546055872 18446744073709551615 47802409586891 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/8045/statm: 78243 15628 1826 364 0 14167 0
Current children cumulated CPU time (s) 1.24
Current children cumulated vsize (KiB) 312972

[startup+3.10024 s]
/proc/loadavg: 7.46 6.07 2.98 2/364 8050
/proc/meminfo: memFree=22252420/32951124 swapFree=67057896/67111528
[pid=8045] ppid=8043 vsize=312972 CPUtime=2.84 cores=0-7
/proc/8045/stat : 8045 (python) R 8043 8045 7833 0 -1 4202496 21966 933 1 0 270 14 0 0 19 0 1 0 648955302 320483328 15628 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440596 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/8045/statm: 78243 15628 1826 364 0 14167 0
Current children cumulated CPU time (s) 2.84
Current children cumulated vsize (KiB) 312972

[startup+6.3007 s]
/proc/loadavg: 7.46 6.07 2.98 2/364 8050
/proc/meminfo: memFree=22252916/32951124 swapFree=67057896/67111528
[pid=8045] ppid=8043 vsize=312972 CPUtime=6.03 cores=0-7
/proc/8045/stat : 8045 (python) R 8043 8045 7833 0 -1 4202496 21973 933 1 0 589 14 0 0 25 0 1 0 648955302 320483328 15635 33554432000 4194304 5683644 140736546055872 18446744073709551615 47802402175116 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/8045/statm: 78243 15635 1833 364 0 14167 0
Current children cumulated CPU time (s) 6.03
Current children cumulated vsize (KiB) 312972

[startup+12.7015 s]
/proc/loadavg: 7.11 6.04 3.00 9/388 8075
/proc/meminfo: memFree=21976532/32951124 swapFree=67057896/67111528
[pid=8045] ppid=8043 vsize=457836 CPUtime=7.44 cores=0-7
/proc/8045/stat : 8045 (python) S 8043 8045 7833 0 -1 4202496 27822 933 1 0 727 17 0 0 25 0 9 0 648955302 468824064 15205 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273443275 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/8045/statm: 114459 15205 1889 364 0 50356 0
[pid=8052] ppid=8045 vsize=310264 CPUtime=0.01 cores=0-7
/proc/8052/stat : 8052 (python) S 8045 8045 7833 0 -1 4202560 894 0 0 0 1 0 0 0 15 0 1 0 648956050 317710336 13495 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636623 0 0 16781312 514 17379716421759500547 0 0 17 1 0 0 0
/proc/8052/statm: 77566 13495 376 364 0 13484 0
[pid=8054] ppid=8052 vsize=268420 CPUtime=5.2 cores=0-7
/proc/8054/stat : 8054 (hybridGM3) R 8052 8054 8054 0 -1 4202496 8087 0 0 0 519 1 0 0 25 0 1 0 648956050 274862080 7823 33554432000 134512640 135204385 4294769968 18446744073709551615 134536698 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/8054/statm: 67105 7823 64 169 0 66935 0
[pid=8055] ppid=8045 vsize=320520 CPUtime=0.01 cores=0-7
/proc/8055/stat : 8055 (python) S 8045 8045 7833 0 -1 4202560 910 0 0 0 1 0 0 0 15 0 1 0 648956053 328212480 13493 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 6103648534823665924 0 0 17 1 0 0 0
/proc/8055/statm: 80130 13493 370 364 0 16045 0
[pid=8057] ppid=8055 vsize=268420 CPUtime=5.17 cores=0-7
/proc/8057/stat : 8057 (hybridGM3) R 8055 8057 8057 0 -1 4202496 8086 0 0 0 517 0 0 0 25 0 1 0 648956053 274862080 7820 33554432000 134512640 135204385 4286980160 18446744073709551615 134545851 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/8057/statm: 67105 7820 64 169 0 66935 0
[pid=8058] ppid=8045 vsize=396312 CPUtime=0.01 cores=0-7
/proc/8058/stat : 8058 (python) S 8045 8045 7833 0 -1 4202560 781 0 0 0 0 1 0 0 15 0 1 0 648956056 405823488 13671 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 15537976720874242307 0 0 17 7 0 0 0
/proc/8058/statm: 99078 13671 371 364 0 34990 0
[pid=8060] ppid=8058 vsize=268420 CPUtime=5.15 cores=0-7
/proc/8060/stat : 8060 (hybridGM3) R 8058 8060 8060 0 -1 4202496 8071 0 0 0 514 1 0 0 25 0 1 0 648956056 274862080 7819 33554432000 134512640 135204385 4286856256 18446744073709551615 134537249 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/8060/statm: 67105 7819 64 169 0 66935 0
[pid=8061] ppid=8045 vsize=406568 CPUtime=0.01 cores=0-7
/proc/8061/stat : 8061 (python) S 8045 8045 7833 0 -1 4202560 795 0 0 0 0 1 0 0 15 0 1 0 648956059 416325632 13632 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 15537965176002150657 0 0 17 0 0 0 0
/proc/8061/statm: 101642 13632 371 364 0 37551 0
[pid=8063] ppid=8061 vsize=268420 CPUtime=5.11 cores=0-7
/proc/8063/stat : 8063 (hybridGM3) R 8061 8063 8063 0 -1 4202496 8081 0 0 0 510 1 0 0 25 0 1 0 648956060 274862080 7828 33554432000 134512640 135204385 4290532656 18446744073709551615 134537272 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/8063/statm: 67105 7828 64 169 0 66935 0
[pid=8064] ppid=8045 vsize=416824 CPUtime=0.01 cores=0-7
/proc/8064/stat : 8064 (python) S 8045 8045 7833 0 -1 4202560 801 0 0 0 0 1 0 0 15 0 1 0 648956062 426827776 13654 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 15537969024292847873 0 0 17 4 0 0 0
/proc/8064/statm: 104206 13654 371 364 0 40112 0
[pid=8066] ppid=8064 vsize=268424 CPUtime=5.05 cores=0-7
/proc/8066/stat : 8066 (hybridGM3) R 8064 8066 8066 0 -1 4202496 8075 0 0 0 504 1 0 0 25 0 1 0 648956063 274866176 7819 33554432000 134512640 135204385 4289089072 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/8066/statm: 67106 7819 64 169 0 66936 0
[pid=8067] ppid=8045 vsize=427080 CPUtime=0.01 cores=0-7
/proc/8067/stat : 8067 (python) S 8045 8045 7833 0 -1 4202560 814 0 0 0 0 1 0 0 15 0 1 0 648956066 437329920 13675 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 8685657528853299458 0 0 17 0 0 0 0
/proc/8067/statm: 106770 13675 371 364 0 42673 0
[pid=8069] ppid=8067 vsize=268424 CPUtime=4.96 cores=0-7
/proc/8069/stat : 8069 (hybridGM3) R 8067 8069 8069 0 -1 4202496 8078 0 0 0 495 1 0 0 25 0 1 0 648956066 274866176 7819 33554432000 134512640 135204385 4287450336 18446744073709551615 134547392 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/8069/statm: 67106 7819 64 169 0 66936 0
[pid=8070] ppid=8045 vsize=437336 CPUtime=0.01 cores=0-7
/proc/8070/stat : 8070 (python) S 8045 8045 7833 0 -1 4202560 820 0 0 0 0 1 0 0 16 0 1 0 648956069 447832064 13696 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 9910246025992306946 0 0 17 4 0 0 0
/proc/8070/statm: 109334 13696 371 364 0 45234 0
[pid=8072] ppid=8070 vsize=268424 CPUtime=4.96 cores=0-7
/proc/8072/stat : 8072 (hybridGM3) R 8070 8072 8072 0 -1 4202496 8081 0 0 0 495 1 0 0 25 0 1 0 648956070 274866176 7820 33554432000 134512640 135204385 4292766544 18446744073709551615 134537383 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/8072/statm: 67106 7820 64 169 0 66936 0
[pid=8073] ppid=8045 vsize=447592 CPUtime=0 cores=0-7
/proc/8073/stat : 8073 (python) S 8045 8045 7833 0 -1 4202560 864 0 0 0 0 0 0 0 15 0 1 0 648956073 458334208 13689 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 17379712573468803332 0 0 17 4 0 0 0
/proc/8073/statm: 111898 13689 377 364 0 47795 0
[pid=8075] ppid=8073 vsize=268424 CPUtime=4.91 cores=0-7
/proc/8075/stat : 8075 (hybridGM3) R 8073 8075 8075 0 -1 4202496 8103 0 0 0 491 0 0 0 25 0 1 0 648956074 274866176 7838 33554432000 134512640 135204385 4290164128 18446744073709551615 134537357 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/8075/statm: 67106 7838 64 169 0 66936 0
[pid=8045/tid=8053] ppid=8043 vsize=457836 CPUtime=0 cores=0-7
/proc/8045/task/8053/stat : 8053 (python) S 8043 8045 7833 0 -1 4202560 25 933 0 0 0 0 0 0 25 0 9 0 648956050 468824064 15205 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8056] ppid=8043 vsize=457836 CPUtime=0 cores=0-7
/proc/8045/task/8056/stat : 8056 (python) S 8043 8045 7833 0 -1 4202560 27 933 0 0 0 0 0 0 19 0 9 0 648956053 468824064 15205 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 3 0 0 0
[pid=8045/tid=8059] ppid=8043 vsize=457836 CPUtime=0 cores=0-7
/proc/8045/task/8059/stat : 8059 (python) S 8043 8045 7833 0 -1 4202560 86 933 0 0 0 0 0 0 19 0 9 0 648956056 468824064 15205 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8062] ppid=8043 vsize=457836 CPUtime=0 cores=0-7
/proc/8045/task/8062/stat : 8062 (python) S 8043 8045 7833 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 9 0 648956059 468824064 15205 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8065] ppid=8043 vsize=457836 CPUtime=0 cores=0-7
/proc/8045/task/8065/stat : 8065 (python) S 8043 8045 7833 0 -1 4202560 34 933 0 0 0 0 0 0 22 0 9 0 648956063 468824064 15205 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8068] ppid=8043 vsize=457836 CPUtime=0 cores=0-7
/proc/8045/task/8068/stat : 8068 (python) S 8043 8045 7833 0 -1 4202560 32 933 0 0 0 0 0 0 24 0 9 0 648956066 468824064 15205 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8071] ppid=8043 vsize=457836 CPUtime=0 cores=0-7
/proc/8045/task/8071/stat : 8071 (python) S 8043 8045 7833 0 -1 4202560 35 933 0 0 0 0 0 0 25 0 9 0 648956070 468824064 15205 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8074] ppid=8043 vsize=457836 CPUtime=0 cores=0-7
/proc/8045/task/8074/stat : 8074 (python) S 8043 8045 7833 0 -1 4202560 33 933 0 0 0 0 0 0 25 0 9 0 648956073 468824064 15205 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 48.02
Current children cumulated vsize (KiB) 5767708

[startup+25.5013 s]
/proc/loadavg: 7.25 6.10 3.05 9/388 8075
/proc/meminfo: memFree=21978808/32951124 swapFree=67057896/67111528
[pid=8045] ppid=8043 vsize=457836 CPUtime=7.44 cores=0-7
/proc/8045/stat : 8045 (python) S 8043 8045 7833 0 -1 4202496 27822 933 1 0 727 17 0 0 25 0 9 0 648955302 468824064 15205 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273443275 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/8045/statm: 114459 15205 1889 364 0 50356 0
[pid=8052] ppid=8045 vsize=310264 CPUtime=0.04 cores=0-7
/proc/8052/stat : 8052 (python) S 8045 8045 7833 0 -1 4202560 900 0 0 0 2 2 0 0 15 0 1 0 648956050 317710336 13495 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636623 0 0 16781312 514 15720377449290105091 0 0 17 1 0 0 0
/proc/8052/statm: 77566 13495 376 364 0 13484 0
[pid=8054] ppid=8052 vsize=268420 CPUtime=18 cores=0-7
/proc/8054/stat : 8054 (hybridGM3) R 8052 8054 8054 0 -1 4202496 8087 0 0 0 1799 1 0 0 25 0 1 0 648956050 274862080 7823 33554432000 134512640 135204385 4294769968 18446744073709551615 134546144 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/8054/statm: 67105 7823 64 169 0 66935 0
[pid=8055] ppid=8045 vsize=320520 CPUtime=0.04 cores=0-7
/proc/8055/stat : 8055 (python) S 8045 8045 7833 0 -1 4202560 916 0 0 0 2 2 0 0 15 0 1 0 648956053 328212480 13493 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 15720377449290105092 0 0 17 1 0 0 0
/proc/8055/statm: 80130 13493 370 364 0 16045 0
[pid=8057] ppid=8055 vsize=268420 CPUtime=17.96 cores=0-7
/proc/8057/stat : 8057 (hybridGM3) R 8055 8057 8057 0 -1 4202496 8090 0 0 0 1796 0 0 0 25 0 1 0 648956053 274862080 7824 33554432000 134512640 135204385 4286980160 18446744073709551615 134546073 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/8057/statm: 67105 7824 64 169 0 66935 0
[pid=8058] ppid=8045 vsize=396312 CPUtime=0.04 cores=0-7
/proc/8058/stat : 8058 (python) S 8045 8045 7833 0 -1 4202560 782 0 0 0 2 2 0 0 15 0 1 0 648956056 405823488 13671 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 15720377449290105091 0 0 17 7 0 0 0
/proc/8058/statm: 99078 13671 371 364 0 34990 0
[pid=8060] ppid=8058 vsize=268420 CPUtime=17.95 cores=0-7
/proc/8060/stat : 8060 (hybridGM3) R 8058 8060 8060 0 -1 4202496 8074 0 0 0 1794 1 0 0 25 0 1 0 648956056 274862080 7822 33554432000 134512640 135204385 4286856256 18446744073709551615 134536646 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/8060/statm: 67105 7822 64 169 0 66935 0
[pid=8061] ppid=8045 vsize=406568 CPUtime=0.04 cores=0-7
/proc/8061/stat : 8061 (python) S 8045 8045 7833 0 -1 4202560 795 0 0 0 2 2 0 0 15 0 1 0 648956059 416325632 13632 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 15720377449290105089 0 0 17 0 0 0 0
/proc/8061/statm: 101642 13632 371 364 0 37551 0
[pid=8063] ppid=8061 vsize=268420 CPUtime=17.91 cores=0-7
/proc/8063/stat : 8063 (hybridGM3) R 8061 8063 8063 0 -1 4202496 8081 0 0 0 1790 1 0 0 25 0 1 0 648956060 274862080 7828 33554432000 134512640 135204385 4290532656 18446744073709551615 134537383 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/8063/statm: 67105 7828 64 169 0 66935 0
[pid=8064] ppid=8045 vsize=416824 CPUtime=0.05 cores=0-7
/proc/8064/stat : 8064 (python) S 8045 8045 7833 0 -1 4202560 802 0 0 0 2 3 0 0 15 0 1 0 648956062 426827776 13654 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 15720377449290105089 0 0 17 4 0 0 0
/proc/8064/statm: 104206 13654 371 364 0 40112 0
[pid=8066] ppid=8064 vsize=268424 CPUtime=17.78 cores=0-7
/proc/8066/stat : 8066 (hybridGM3) R 8064 8066 8066 0 -1 4202496 8077 0 0 0 1777 1 0 0 25 0 1 0 648956063 274866176 7821 33554432000 134512640 135204385 4289089072 18446744073709551615 134546153 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/8066/statm: 67106 7821 64 169 0 66936 0

################
# More data... #
################

/proc/8147/task/8169/stat : 8169 (python) S 8045 8045 7833 0 -1 4202560 83 0 0 0 0 0 0 0 19 0 2 0 649025479 615866368 14081 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 514 18446604441448908672 0 0 -1 4 0 0 0
[pid=8151] ppid=8045 vsize=611688 CPUtime=0.59 cores=0-7
/proc/8151/stat : 8151 (python) S 8045 8045 7833 0 -1 4202560 1187 0 0 0 33 26 0 0 16 0 2 0 649005829 626368512 14035 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/8151/statm: 152922 14035 407 364 0 88771 0
[pid=8153] ppid=8151 vsize=268424 CPUtime=196.1 cores=0-7
/proc/8153/stat : 8153 (hybridGM3) T 8151 8153 8153 0 -1 4202496 8110 0 0 0 19609 1 0 0 25 0 1 0 649005830 274866176 7826 33554432000 134512640 135204385 4291631088 18446744073709551615 134537383 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/8153/statm: 67106 7826 64 169 0 66936 0
[pid=8151/tid=8173] ppid=8045 vsize=611688 CPUtime=0 cores=0-7
/proc/8151/task/8173/stat : 8173 (python) S 8045 8045 7833 0 -1 4202560 82 0 0 0 0 0 0 0 16 0 2 0 649025507 626368512 14035 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 514 18446604453903594496 0 0 -1 1 0 0 0
[pid=8157] ppid=8045 vsize=621944 CPUtime=0.29 cores=0-7
/proc/8157/stat : 8157 (python) S 8045 8045 7833 0 -1 4202560 1180 0 0 0 14 15 0 0 19 0 2 0 649014886 636870656 14038 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/8157/statm: 155486 14038 400 364 0 91332 0
[pid=8159] ppid=8157 vsize=465044 CPUtime=99.87 cores=0-7
/proc/8159/stat : 8159 (TNM) T 8157 8159 8159 0 -1 4202496 6035 0 0 0 9986 1 0 0 25 0 1 0 649014887 476205056 5750 33554432000 134512640 135008748 4291453024 18446744073709551615 134529448 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/8159/statm: 116261 5750 46 122 0 115161 0
[pid=8157/tid=8165] ppid=8045 vsize=621944 CPUtime=0 cores=0-7
/proc/8157/task/8165/stat : 8165 (python) S 8045 8045 7833 0 -1 4202560 86 0 0 0 0 0 0 0 19 0 2 0 649024916 636870656 14038 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 514 18446604444759362368 0 0 -1 4 0 0 0
[pid=8166] ppid=8045 vsize=632200 CPUtime=0.65 cores=0-7
/proc/8166/stat : 8166 (python) S 8045 8045 7833 0 -1 4202560 1192 0 0 0 31 34 0 0 18 0 2 0 649024924 647372800 14064 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/8166/statm: 158050 14064 400 364 0 93893 0
[pid=8168] ppid=8166 vsize=431704 CPUtime=198.45 cores=0-7
/proc/8168/stat : 8168 (adaptg2wsat2009) T 8166 8168 8168 0 -1 4202496 6268 0 0 0 19844 1 0 0 25 0 1 0 649024925 442064896 5983 33554432000 134512640 135107411 4290364320 18446744073709551615 134531795 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/8168/statm: 107926 5983 59 146 0 106802 0
[pid=8166/tid=8202] ppid=8045 vsize=632200 CPUtime=0 cores=0-7
/proc/8166/task/8202/stat : 8202 (python) S 8045 8045 7833 0 -1 4202560 87 0 0 0 0 0 0 0 19 0 2 0 649044894 647372800 14064 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 514 18446604467373831360 0 0 -1 4 0 0 0
[pid=8170] ppid=8045 vsize=642456 CPUtime=0.6 cores=0-7
/proc/8170/stat : 8170 (python) S 8045 8045 7833 0 -1 4202560 991 0 0 0 30 30 0 0 15 0 1 0 649025486 657874944 14092 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 15720377449290105094 0 0 17 7 0 0 0
/proc/8170/statm: 160614 14092 372 364 0 96454 0
[pid=8172] ppid=8170 vsize=268420 CPUtime=194.19 cores=0-7
/proc/8172/stat : 8172 (hybridGM3) R 8170 8172 8172 0 -1 4202496 8108 0 0 0 19417 2 0 0 25 0 1 0 649025486 274862080 7823 33554432000 134512640 135204385 4290081088 18446744073709551615 134536676 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/8172/statm: 67105 7823 64 169 0 66935 0
[pid=8174] ppid=8045 vsize=652712 CPUtime=0.32 cores=0-7
/proc/8174/stat : 8174 (python) S 8045 8045 7833 0 -1 4202560 1200 0 0 0 15 17 0 0 19 0 2 0 649025514 668377088 14114 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/8174/statm: 163178 14114 401 364 0 99015 0
[pid=8176] ppid=8174 vsize=465044 CPUtime=98.97 cores=0-7
/proc/8176/stat : 8176 (TNM) T 8174 8176 8176 0 -1 4202496 6034 0 0 0 9897 0 0 0 25 0 1 0 649025515 476205056 5749 33554432000 134512640 135008748 4294681056 18446744073709551615 134527542 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/8176/statm: 116261 5749 46 122 0 115161 0
[pid=8174/tid=8190] ppid=8045 vsize=652712 CPUtime=0 cores=0-7
/proc/8174/task/8190/stat : 8190 (python) S 8045 8045 7833 0 -1 4202560 82 0 0 0 0 0 0 0 19 0 2 0 649035447 668377088 14114 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 514 18446604463253787904 0 0 -1 2 0 0 0
[pid=8178] ppid=8045 vsize=662968 CPUtime=0.32 cores=0-7
/proc/8178/stat : 8178 (python) S 8045 8045 7833 0 -1 4202560 1241 0 0 0 14 18 0 0 19 0 2 0 649025709 678879232 14099 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8178/statm: 165742 14099 406 364 0 101576 0
[pid=8180] ppid=8178 vsize=465044 CPUtime=100.34 cores=0-7
/proc/8180/stat : 8180 (TNM) T 8178 8180 8180 0 -1 4202496 6036 0 0 0 10033 1 0 0 25 0 1 0 649025709 476205056 5749 33554432000 134512640 135008748 4287012544 18446744073709551615 134527111 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/8180/statm: 116261 5749 46 122 0 115161 0
[pid=8178/tid=8191] ppid=8045 vsize=662968 CPUtime=0 cores=0-7
/proc/8178/task/8191/stat : 8191 (python) S 8045 8045 7833 0 -1 4202560 85 0 0 0 0 0 0 0 19 0 2 0 649035744 678879232 14099 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 514 18446604461411260224 0 0 -1 0 0 0 0
[pid=8181] ppid=8045 vsize=673224 CPUtime=0.34 cores=0-7
/proc/8181/stat : 8181 (python) S 8045 8045 7833 0 -1 4202560 1237 0 0 0 17 17 0 0 19 0 2 0 649025720 689381376 14193 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8181/statm: 168306 14193 406 364 0 104137 0
[pid=8183] ppid=8181 vsize=465044 CPUtime=98.48 cores=0-7
/proc/8183/stat : 8183 (TNM) T 8181 8183 8183 0 -1 4202496 6041 0 0 0 9845 3 0 0 25 0 1 0 649025720 476205056 5749 33554432000 134512640 135008748 4286571744 18446744073709551615 134527773 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/8183/statm: 116261 5749 46 122 0 115161 0
[pid=8181/tid=8192] ppid=8045 vsize=673224 CPUtime=0 cores=0-7
/proc/8181/task/8192/stat : 8192 (python) S 8045 8045 7833 0 -1 4202560 83 0 0 0 0 0 0 0 19 0 2 0 649036160 689381376 14193 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 514 18446604471112304768 0 0 -1 0 0 0 0
[pid=8186] ppid=8045 vsize=683480 CPUtime=0.32 cores=0-7
/proc/8186/stat : 8186 (python) S 8045 8045 7833 0 -1 4202560 1254 0 0 0 15 17 0 0 19 0 2 0 649034446 699883520 14149 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8186/statm: 170870 14149 407 364 0 106698 0
[pid=8188] ppid=8186 vsize=465044 CPUtime=99.7 cores=0-7
/proc/8188/stat : 8188 (TNM) T 8186 8188 8188 0 -1 4202496 6039 0 0 0 9969 1 0 0 25 0 1 0 649034447 476205056 5750 33554432000 134512640 135008748 4292113904 18446744073709551615 134527729 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/8188/statm: 116261 5750 46 122 0 115161 0
[pid=8186/tid=8198] ppid=8045 vsize=683480 CPUtime=0 cores=0-7
/proc/8186/task/8198/stat : 8198 (python) S 8045 8045 7833 0 -1 4202560 88 0 0 0 0 0 0 0 19 0 2 0 649044486 699883520 14149 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 514 18446604471126989824 0 0 -1 2 0 0 0
[pid=8193] ppid=8045 vsize=693736 CPUtime=0.29 cores=0-7
/proc/8193/stat : 8193 (python) S 8045 8045 7833 0 -1 4202560 1064 0 0 0 13 16 0 0 18 0 1 0 649036167 710385664 14070 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 15720377449290105092 0 0 17 1 0 0 0
/proc/8193/statm: 173434 14070 379 364 0 109259 0
[pid=8195] ppid=8193 vsize=465044 CPUtime=87.22 cores=0-7
/proc/8195/stat : 8195 (TNM) R 8193 8195 8195 0 -1 4202496 6041 0 0 0 8722 0 0 0 25 0 1 0 649036168 476205056 5749 33554432000 134512640 135008748 4291951872 18446744073709551615 134527416 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/8195/statm: 116261 5749 46 122 0 115161 0
[pid=8199] ppid=8045 vsize=703992 CPUtime=0.01 cores=0-7
/proc/8199/stat : 8199 (python) S 8045 8045 7833 0 -1 4202560 1063 0 0 0 1 0 0 0 18 0 1 0 649044493 720887808 14168 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 16346350588876783878 0 0 17 0 0 0 0
/proc/8199/statm: 175998 14168 378 364 0 111820 0
[pid=8201] ppid=8199 vsize=465044 CPUtime=4.48 cores=0-7
/proc/8201/stat : 8201 (TNM) R 8199 8201 8201 0 -1 4202496 6039 0 0 0 447 1 0 0 25 0 1 0 649044494 476205056 5750 33554432000 134512640 135008748 4289229520 18446744073709551615 134529480 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/8201/statm: 116261 5750 46 122 0 115161 0
[pid=8203] ppid=8045 vsize=714248 CPUtime=0 cores=0-7
/proc/8203/stat : 8203 (python) S 8045 8045 7833 0 -1 4202560 927 0 0 0 0 0 0 0 19 0 1 0 649044901 731389952 14052 33554432000 4194304 5683644 140736546055872 18446744073709551615 254261636678 0 0 16781312 514 10740206880660914175 0 0 17 5 0 0 0
/proc/8203/statm: 178562 14052 346 364 0 114381 0
[pid=8205] ppid=8203 vsize=461348 CPUtime=0.42 cores=0-7
/proc/8205/stat : 8205 (TNM) R 8203 8205 8205 0 -1 4202496 3713 0 0 0 42 0 0 0 21 0 1 0 649044902 472420352 3425 33554432000 134512640 135008748 4286900496 18446744073709551615 134522330 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/8205/statm: 115337 3425 45 122 0 114237 0
[pid=8045/tid=8053] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8053/stat : 8053 (python) S 8043 8045 7833 0 -1 4202560 31 933 0 0 0 0 0 0 15 0 35 0 648956050 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8056] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8056/stat : 8056 (python) S 8043 8045 7833 0 -1 4202560 35 933 0 0 0 0 0 0 18 0 35 0 648956053 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 3 0 0 0
[pid=8045/tid=8059] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8059/stat : 8059 (python) S 8043 8045 7833 0 -1 4202560 95 933 0 0 0 0 0 0 18 0 35 0 648956056 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8062] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8062/stat : 8062 (python) S 8043 8045 7833 0 -1 4202560 49 933 0 0 0 0 0 0 16 0 35 0 648956059 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 6 0 0 0
[pid=8045/tid=8065] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8065/stat : 8065 (python) S 8043 8045 7833 0 -1 4202560 47 933 0 0 0 0 0 0 18 0 35 0 648956063 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8068] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8068/stat : 8068 (python) S 8043 8045 7833 0 -1 4202560 43 933 0 0 0 0 0 0 18 0 35 0 648956066 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8071] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8071/stat : 8071 (python) S 8043 8045 7833 0 -1 4202560 40 933 0 0 0 0 0 0 18 0 35 0 648956070 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8074] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8074/stat : 8074 (python) S 8043 8045 7833 0 -1 4202560 38 933 0 0 0 0 0 0 18 0 35 0 648956073 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
[pid=8045/tid=8088] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8088/stat : 8088 (python) S 8043 8045 7833 0 -1 4202560 44 933 0 0 0 0 0 0 19 0 35 0 648975628 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 5 0 0 0
[pid=8045/tid=8096] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8096/stat : 8096 (python) S 8043 8045 7833 0 -1 4202560 62 933 0 0 0 0 0 0 18 0 35 0 648985663 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 7 0 0 0
[pid=8045/tid=8099] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8099/stat : 8099 (python) S 8043 8045 7833 0 -1 4202560 37 933 0 0 0 0 0 0 18 0 35 0 648985669 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 7 0 0 0
[pid=8045/tid=8102] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8102/stat : 8102 (python) S 8043 8045 7833 0 -1 4202560 34 933 0 0 0 0 0 0 19 0 35 0 648985750 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 1 0 0 0
[pid=8045/tid=8105] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8105/stat : 8105 (python) S 8043 8045 7833 0 -1 4202560 35 933 0 0 0 0 0 0 21 0 35 0 648985754 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
[pid=8045/tid=8108] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8108/stat : 8108 (python) S 8043 8045 7833 0 -1 4202560 39 933 0 0 0 0 0 0 22 0 35 0 648985771 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
[pid=8045/tid=8111] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8111/stat : 8111 (python) S 8043 8045 7833 0 -1 4202560 35 933 0 0 0 0 0 0 20 0 35 0 648985970 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
[pid=8045/tid=8117] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8117/stat : 8117 (python) S 8043 8045 7833 0 -1 4202560 36 933 0 0 0 0 0 0 20 0 35 0 648995388 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 1 0 0 0
[pid=8045/tid=8121] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8121/stat : 8121 (python) S 8043 8045 7833 0 -1 4202560 31 933 0 0 0 0 0 0 19 0 35 0 648995785 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
[pid=8045/tid=8125] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8125/stat : 8125 (python) S 8043 8045 7833 0 -1 4202560 33 933 0 0 0 0 0 0 19 0 35 0 648995792 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
[pid=8045/tid=8129] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8129/stat : 8129 (python) S 8043 8045 7833 0 -1 4202560 33 933 0 0 0 0 0 0 20 0 35 0 648995805 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
[pid=8045/tid=8133] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8133/stat : 8133 (python) S 8043 8045 7833 0 -1 4202560 39 933 0 0 0 0 0 0 18 0 35 0 648996004 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
[pid=8045/tid=8139] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8139/stat : 8139 (python) S 8043 8045 7833 0 -1 4202560 37 933 0 0 0 0 0 0 18 0 35 0 649004826 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 1 0 0 0
[pid=8045/tid=8143] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8143/stat : 8143 (python) S 8043 8045 7833 0 -1 4202560 33 933 0 0 0 0 0 0 20 0 35 0 649005427 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
[pid=8045/tid=8148] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8148/stat : 8148 (python) S 8043 8045 7833 0 -1 4202560 41 933 0 0 0 0 0 0 18 0 35 0 649005822 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
[pid=8045/tid=8152] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8152/stat : 8152 (python) S 8043 8045 7833 0 -1 4202560 31 933 0 0 0 0 0 0 21 0 35 0 649005830 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
[pid=8045/tid=8158] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8158/stat : 8158 (python) S 8043 8045 7833 0 -1 4202560 34 933 0 0 0 0 0 0 20 0 35 0 649014887 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 1 0 0 0
[pid=8045/tid=8167] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8167/stat : 8167 (python) S 8043 8045 7833 0 -1 4202560 38 933 0 0 0 0 0 0 20 0 35 0 649024925 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8171] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8171/stat : 8171 (python) S 8043 8045 7833 0 -1 4202560 34 933 0 0 0 0 0 0 20 0 35 0 649025486 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 7 0 0 0
[pid=8045/tid=8175] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8175/stat : 8175 (python) S 8043 8045 7833 0 -1 4202560 33 933 0 0 0 0 0 0 20 0 35 0 649025515 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 7 0 0 0
[pid=8045/tid=8179] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8179/stat : 8179 (python) S 8043 8045 7833 0 -1 4202560 38 933 0 0 0 0 0 0 20 0 35 0 649025709 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 7 0 0 0
[pid=8045/tid=8182] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8182/stat : 8182 (python) S 8043 8045 7833 0 -1 4202560 32 933 0 0 0 0 0 0 21 0 35 0 649025720 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 7 0 0 0
[pid=8045/tid=8187] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8187/stat : 8187 (python) S 8043 8045 7833 0 -1 4202560 33 933 0 0 0 0 0 0 20 0 35 0 649034447 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 0 0 0 0
[pid=8045/tid=8194] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8194/stat : 8194 (python) S 8043 8045 7833 0 -1 4202560 34 933 0 0 0 0 0 0 20 0 35 0 649036168 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 1 0 0 0
[pid=8045/tid=8200] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8200/stat : 8200 (python) S 8043 8045 7833 0 -1 4202560 34 933 0 0 0 0 0 0 19 0 35 0 649044494 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 1 0 0 0
[pid=8045/tid=8204] ppid=8043 vsize=724492 CPUtime=0 cores=0-7
/proc/8045/task/8204/stat : 8204 (python) S 8043 8045 7833 0 -1 4202560 34 933 0 0 0 0 0 0 19 0 35 0 649044901 741879808 15652 33554432000 4194304 5683644 140736546055872 18446744073709551615 254273440177 0 0 16781312 2 18446604444074320128 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 7086.74
Current children cumulated vsize (KiB) 31926284

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 8045 and gives
#  childrusage.ru_utime.tv_sec=1479
#  childrusage.ru_utime.tv_usec=343105
#  childrusage.ru_stime.tv_sec=11
#  childrusage.ru_stime.tv_usec=182300
# CPU time returned by wait4() is 1490.53
# while last known CPU time is 7086.74
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 896.674
CPU time (s): 7086.74
CPU user time (s): 7075.62
CPU system time (s): 11.12
CPU usage (%): 790.336
Max. virtual memory (cumulated for all children) (KiB): 31926284

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1479.34
system time used= 11.1823
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 140475
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 12311
involuntary context switches= 9909

runsolver used 8.46771 second user time and 21.5547 second system time

The end

Launcher Data 

Begin job on node140 at 2011-05-16 19:03:20
IDJOB=3385344
IDBENCH=82609
IDSOLVER=1870
FILE ID=node140/3385344-1305565399
RUNJOBID= node140-1305564961-7847
PBS_JOBID= 13350110
Free space on /tmp= 73440 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v35000-c147000-S648642831-094-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385344-1305565399/watcher-3385344-1305565399 -o /tmp/evaluation-result-3385344-1305565399/solver-3385344-1305565399 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385344-1305565399.cnf -seed 1889413889 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 4a57833314362e0939d4095b3206ad82
RANDOM SEED=1889413889

node140.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      22309004 kB
Buffers:        330596 kB
Cached:        4922124 kB
SwapCached:       8088 kB
Active:        6921120 kB
Inactive:      3387220 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      22309004 kB
SwapTotal:    67111528 kB
SwapFree:     67057896 kB
Dirty:           34596 kB
Writeback:           0 kB
AnonPages:     5052404 kB
Mapped:          29452 kB
Slab:           228048 kB
PageTables:      35676 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 91921892 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73408 MiB
End job on node140 at 2011-05-16 19:18:18