Trace number 3321923

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 7924.96 1004.41

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v50000-c210000-S964643921-035-UNKNOWN.cnf
MD5SUMd8337de8a71a630ccf2dd02943cd7d7f
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark801.476
Satisfiable
(Un)Satisfiability was proved
Number of variables50000
Number of clauses210000
Sum of the clauses size630000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3210000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.35/0.68	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.68	c INFO: solving HOME/instance-3321923-1304369577.cnf
0.35/0.68	c INFO: computing task features
1.13/1.40	c INFO: computed variable-clause graph statistics
1.13/1.43	c INFO: computed clause balance statistics
4.73/5.03	c INFO: computed variable balance statistics
7.63/7.90	c INFO: computed small-clause counts
7.63/7.92	c INFO: computed Horn-clause counts
10.64/11.00	c INFO: computed variable graph statistics
10.64/11.00	c INFO: collected features for HOME/instance-3321923-1304369577.cnf in 10.29 s
10.74/11.03	c INFO: running hybridGM3@0 for 97 with 39989 remaining (b = 0.35)
10.74/11.08	c INFO: running hybridGM3@0 for 97 with 39989 remaining (b = 0.21)
10.82/11.12	c INFO: running hybridGM3@0 for 97 with 39989 remaining (b = 0.14)
10.82/11.16	c INFO: running hybridGM3@0 for 194 with 39989 remaining (b = 0.18)
10.82/11.19	c INFO: running hybridGM3@0 for 194 with 39989 remaining (b = 0.13)
10.92/11.24	c INFO: running hybridGM3@0 for 194 with 39989 remaining (b = 0.10)
10.92/11.28	c INFO: running hybridGM3@0 for 194 with 39989 remaining (b = 0.07)
11.02/11.32	c INFO: running TNM@0 for 97 with 39989 remaining (b = 0.03)
811.22/111.42	c INFO: running hybridGM3@0 for 97 with 39889 remaining (b = 0.03)
811.22/111.46	c INFO: running hybridGM3@0 for 97 with 39889 remaining (b = 0.03)
811.97/111.50	c INFO: running hybridGM3@0 for 97 with 39889 remaining (b = 0.03)
811.97/111.57	c INFO: running hybridGM3@0 for 194 with 39889 remaining (b = 0.05)
1570.96/206.67	c INFO: running hybridGM3@100 for 97 with 39794 remaining (b = 0.02)
1571.80/206.71	c INFO: running hybridGM3@100 for 97 with 39794 remaining (b = 0.02)
1571.80/206.75	c INFO: running hybridGM3@99 for 97 with 39793 remaining (b = 0.02)
1579.71/207.71	c INFO: running hybridGM3@0 for 97 with 39793 remaining (b = 0.02)
1579.71/207.78	c INFO: running hybridGM3@0 for 97 with 39792 remaining (b = 0.02)
1580.55/207.87	c INFO: running hybridGM3@0 for 97 with 39792 remaining (b = 0.02)
1628.35/213.84	c INFO: running hybridGM3@0 for 97 with 39786 remaining (b = 0.02)
2339.42/302.95	c INFO: running hybridGM3@0 for 194 with 39697 remaining (b = 0.04)
2379.34/307.93	c INFO: running hybridGM3@0 for 194 with 39692 remaining (b = 0.03)
2379.34/307.98	c INFO: running hybridGM3@0 for 194 with 39692 remaining (b = 0.03)
2379.96/308.04	c INFO: running hybridGM3@296 for 97 with 39692 remaining (b = 0.01)
2381.11/308.28	c INFO: running TNM@0 for 97 with 39692 remaining (b = 0.01)
2387.98/309.04	c INFO: running hybridGM3@94 for 97 with 39691 remaining (b = 0.01)
2387.98/309.09	c INFO: running hybridGM3@94 for 97 with 39691 remaining (b = 0.01)
2388.78/309.14	c INFO: running TNM@100 for 1940 with 39691 remaining (b = 0.20)
3140.38/403.30	c INFO: running hybridGM3@0 for 194 with 39597 remaining (b = 0.02)
3148.35/404.36	c INFO: running hybridGM3@0 for 194 with 39596 remaining (b = 0.02)
3182.61/408.67	c INFO: running adaptg2wsat2009++@0 for 194 with 39592 remaining (b = 0.02)
3196.20/410.32	c INFO: running hybridGM3@189 for 97 with 39590 remaining (b = 0.01)
3907.87/499.66	c INFO: running hybridGM3@100 for 97 with 39501 remaining (b = 0.01)
3946.96/504.59	c INFO: running hybridGM3@101 for 97 with 39496 remaining (b = 0.01)
3947.76/504.68	c INFO: running hybridGM3@101 for 97 with 39496 remaining (b = 0.01)
3954.95/505.59	c INFO: running hybridGM3@385 for 97 with 39495 remaining (b = 0.01)
4667.52/594.93	c INFO: running hybridGM3@297 for 97 with 39405 remaining (b = 0.01)
4714.52/600.86	c INFO: running hybridGM3@480 for 97 with 39399 remaining (b = 0.01)
4716.11/601.07	c INFO: running TNM@0 for 97 with 39399 remaining (b = 0.01)
4748.76/605.18	c INFO: running hybridGM3@98 for 97 with 39395 remaining (b = 0.01)
4754.31/605.87	c INFO: running hybridGM3@296 for 97 with 39394 remaining (b = 0.01)
4755.11/605.97	c INFO: running TNM@0 for 97 with 39394 remaining (b = 0.01)
4755.95/606.10	c INFO: running hybridGM3@94 for 97 with 39394 remaining (b = 0.01)
5474.88/696.22	c INFO: running hybridGM3@391 for 97 with 39304 remaining (b = 0.01)
5513.78/701.15	c INFO: running hybridGM3@396 for 97 with 39299 remaining (b = 0.01)
5515.35/701.37	c INFO: running TNM@0 for 97 with 39299 remaining (b = 0.01)
5516.05/701.46	c INFO: running hybridGM3@193 for 97 with 39299 remaining (b = 0.01)
5521.52/702.16	c INFO: running TNM@0 for 97 with 39298 remaining (b = 0.01)
5523.65/702.48	c INFO: running TNM@0 for 97 with 39298 remaining (b = 0.01)
5546.79/705.34	c INFO: running TNM@0 for 97 with 39295 remaining (b = 0.01)
6280.97/797.54	c INFO: running TNM@0 for 97 with 39203 remaining (b = 0.01)
6314.31/801.77	c INFO: running hybridGM3@0 for 194 with 39198 remaining (b = 0.01)
6315.15/801.88	c INFO: running adaptg2wsat2009++@101 for 97 with 39198 remaining (b = 0.01)
6319.84/802.48	c INFO: running TNM@0 for 97 with 39198 remaining (b = 0.01)
6320.53/802.58	c INFO: running TNM@0 for 97 with 39198 remaining (b = 0.01)
6321.75/802.77	c INFO: running hybridGM3@293 for 194 with 39197 remaining (b = 0.01)
6345.14/805.70	c INFO: running TNM@0 for 97 with 39195 remaining (b = 0.01)
7079.00/897.89	c INFO: running adaptg2wsat2009++@0 for 194 with 39102 remaining (b = 0.01)
7090.14/899.22	c INFO: running TNM@0 for 97 with 39101 remaining (b = 0.01)
7111.68/901.98	c INFO: running TNM@0 for 97 with 39098 remaining (b = 0.01)
7118.65/902.94	c INFO: running TNM@0 for 97 with 39097 remaining (b = 0.01)
7137.01/905.11	c INFO: running TNM@0 for 97 with 39095 remaining (b = 0.01)
7880.35/998.67	c INFO: running hybridGM3@98 for 97 with 39002 remaining (b = 0.01)
7887.50/999.59	c INFO: running TNM@0 for 97 with 39001 remaining (b = 0.01)
7910.57/1002.45	c INFO: running hybridGM3@196 for 291 with 38998 remaining (b = 0.02)
7917.63/1003.31	c INFO: running TNM@0 for 97 with 38997 remaining (b = 0.01)
7924.96/1004.39	c INFO: running TNM@0 for 97 with 38996 remaining (b = 0.00)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321923-1304369577/watcher-3321923-1304369577 -o /tmp/evaluation-result-3321923-1304369577/solver-3321923-1304369577 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321923-1304369577.cnf -seed 992975932 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.02 7.97 6.83 2/281 22491
/proc/meminfo: memFree=23343608/32950928 swapFree=54880832/67111528
[pid=22491] ppid=22489 vsize=15752 CPUtime=0 cores=0-7
/proc/22491/stat : 22491 (python) S 22489 22491 21696 0 -1 4202496 840 0 0 0 0 0 0 0 22 0 1 0 548554426 16130048 723 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500689920 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/22491/statm: 3938 723 334 364 0 425 0

[startup+0.0386551 s]
/proc/loadavg: 8.02 7.97 6.83 2/281 22491
/proc/meminfo: memFree=23343608/32950928 swapFree=54880832/67111528
[pid=22491] ppid=22489 vsize=18888 CPUtime=0.01 cores=0-7
/proc/22491/stat : 22491 (python) R 22489 22491 21696 0 -1 4202496 1252 0 0 0 1 0 0 0 22 0 1 0 548554426 19341312 1106 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500691712 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/22491/statm: 4722 1111 421 364 0 691 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18888

[startup+0.100625 s]
/proc/loadavg: 8.02 7.97 6.83 2/281 22491
/proc/meminfo: memFree=23343608/32950928 swapFree=54880832/67111528
[pid=22491] ppid=22489 vsize=87704 CPUtime=0.07 cores=0-7
/proc/22491/stat : 22491 (python) R 22489 22491 21696 0 -1 4202496 2638 0 0 0 5 2 0 0 18 0 1 0 548554426 89808896 2354 33554432000 4194304 5683644 140735917409424 18446744073709551615 47376405959360 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/22491/statm: 21926 2354 717 364 0 1622 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 87704

[startup+0.300616 s]
/proc/loadavg: 8.02 7.97 6.83 2/281 22491
/proc/meminfo: memFree=23343608/32950928 swapFree=54880832/67111528
[pid=22491] ppid=22489 vsize=135748 CPUtime=0.19 cores=0-7
/proc/22491/stat : 22491 (python) S 22489 22491 21696 0 -1 4202496 5963 934 1 0 15 4 0 0 17 0 1 0 548554426 139005952 4151 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500687877 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/22491/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135748

[startup+0.700513 s]
/proc/loadavg: 8.02 7.97 6.83 2/281 22491
/proc/meminfo: memFree=23343608/32950928 swapFree=54880832/67111528
[pid=22491] ppid=22489 vsize=283472 CPUtime=0.43 cores=0-7
/proc/22491/stat : 22491 (python) R 22489 22491 21696 0 -1 4202496 10510 934 1 0 34 9 0 0 16 0 1 0 548554426 290275328 8192 33554432000 4194304 5683644 140735917409424 18446744073709551615 4272792 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/22491/statm: 70868 8193 1761 364 0 6792 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 283472

[startup+1.50034 s]
/proc/loadavg: 7.46 7.85 6.80 2/282 22496
/proc/meminfo: memFree=23297596/32950928 swapFree=54880832/67111528
[pid=22491] ppid=22489 vsize=328000 CPUtime=1.23 cores=0-7
/proc/22491/stat : 22491 (python) R 22489 22491 21696 0 -1 4202496 27454 934 1 0 111 12 0 0 17 0 1 0 548554426 335872000 19385 33554432000 4194304 5683644 140735917409424 18446744073709551615 4943918 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/22491/statm: 82000 19385 1826 364 0 17924 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 328000

[startup+3.10103 s]
/proc/loadavg: 7.46 7.85 6.80 2/282 22496
/proc/meminfo: memFree=23272176/32950928 swapFree=54880832/67111528
[pid=22491] ppid=22489 vsize=328000 CPUtime=2.83 cores=0-7
/proc/22491/stat : 22491 (python) R 22489 22491 21696 0 -1 4202496 27454 934 1 0 271 12 0 0 19 0 1 0 548554426 335872000 19385 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500340946 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/22491/statm: 82000 19385 1826 364 0 17924 0
Current children cumulated CPU time (s) 2.83
Current children cumulated vsize (KiB) 328000

[startup+6.30039 s]
/proc/loadavg: 6.94 7.74 6.77 2/282 22496
/proc/meminfo: memFree=23273176/32950928 swapFree=54880832/67111528
[pid=22491] ppid=22489 vsize=328000 CPUtime=6.03 cores=0-7
/proc/22491/stat : 22491 (python) R 22489 22491 21696 0 -1 4202496 27455 934 1 0 591 12 0 0 25 0 1 0 548554426 335872000 19386 33554432000 4194304 5683644 140735917409424 18446744073709551615 4539162 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/22491/statm: 82000 19386 1827 364 0 17924 0
Current children cumulated CPU time (s) 6.03
Current children cumulated vsize (KiB) 328000

[startup+12.701 s]
/proc/loadavg: 6.46 7.63 6.73 9/306 22520
/proc/meminfo: memFree=23096572/32950928 swapFree=54880832/67111528
[pid=22491] ppid=22489 vsize=471692 CPUtime=11.06 cores=0-7
/proc/22491/stat : 22491 (python) S 22489 22491 21696 0 -1 4202496 34005 934 1 0 1090 16 0 0 20 0 9 0 548554426 483012608 18650 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512518603 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22491/statm: 117923 18650 1889 364 0 53820 0
[pid=22497] ppid=22491 vsize=324120 CPUtime=0 cores=0-7
/proc/22497/stat : 22497 (python) S 22491 22491 21696 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 548555530 331898880 16936 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500711951 0 0 16781312 514 15457861905628004351 0 0 17 6 0 0 0
/proc/22497/statm: 81030 16936 353 364 0 16948 0
[pid=22499] ppid=22497 vsize=276232 CPUtime=1.64 cores=0-7
/proc/22499/stat : 22499 (hybridGM3) R 22497 22499 22499 0 -1 4202496 4995 0 0 0 164 0 0 0 25 0 1 0 548555530 282861568 4728 33554432000 134512640 135204385 4288671952 18446744073709551615 134543975 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22499/statm: 69058 4728 57 169 0 68888 0
[pid=22500] ppid=22491 vsize=334376 CPUtime=0 cores=0-7
/proc/22500/stat : 22500 (python) S 22491 22491 21696 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 548555534 342401024 16934 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 4161603136915505151 0 0 17 0 0 0 0
/proc/22500/statm: 83594 16934 347 364 0 19509 0
[pid=22502] ppid=22500 vsize=276232 CPUtime=1.61 cores=0-7
/proc/22502/stat : 22502 (hybridGM3) R 22500 22502 22502 0 -1 4202496 4959 0 0 0 160 1 0 0 25 0 1 0 548555534 282861568 4693 33554432000 134512640 135204385 4288556448 18446744073709551615 134543968 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/22502/statm: 69058 4693 57 169 0 68888 0
[pid=22503] ppid=22491 vsize=344632 CPUtime=0 cores=0-7
/proc/22503/stat : 22503 (python) S 22491 22491 21696 0 -1 4202560 744 0 0 0 0 0 0 0 18 0 1 0 548555538 352903168 16938 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 1793247668982513663 0 0 17 1 0 0 0
/proc/22503/statm: 86158 16938 347 364 0 22070 0
[pid=22505] ppid=22503 vsize=276240 CPUtime=1.56 cores=0-7
/proc/22505/stat : 22505 (hybridGM3) R 22503 22505 22505 0 -1 4202496 4922 0 0 0 155 1 0 0 25 0 1 0 548555538 282869760 4661 33554432000 134512640 135204385 4288886592 18446744073709551615 134543968 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/22505/statm: 69060 4661 57 169 0 68890 0
[pid=22506] ppid=22491 vsize=354888 CPUtime=0 cores=0-7
/proc/22506/stat : 22506 (python) S 22491 22491 21696 0 -1 4202560 749 0 0 0 0 0 0 0 18 0 1 0 548555542 363405312 16942 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 1079985679951200255 0 0 17 1 0 0 0
/proc/22506/statm: 88722 16942 347 364 0 24631 0
[pid=22508] ppid=22506 vsize=276236 CPUtime=1.52 cores=0-7
/proc/22508/stat : 22508 (hybridGM3) R 22506 22508 22508 0 -1 4202496 4909 0 0 0 152 0 0 0 25 0 1 0 548555542 282865664 4646 33554432000 134512640 135204385 4289990240 18446744073709551615 134543975 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22508/statm: 69059 4646 57 169 0 68889 0
[pid=22509] ppid=22491 vsize=430680 CPUtime=0 cores=0-7
/proc/22509/stat : 22509 (python) S 22491 22491 21696 0 -1 4202560 763 0 0 0 0 0 0 0 18 0 1 0 548555546 441016320 16948 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 12991701854747361279 0 0 17 1 0 0 0
/proc/22509/statm: 107670 16948 347 364 0 43576 0
[pid=22511] ppid=22509 vsize=276236 CPUtime=1.48 cores=0-7
/proc/22511/stat : 22511 (hybridGM3) R 22509 22511 22511 0 -1 4202496 4870 0 0 0 147 1 0 0 25 0 1 0 548555546 282865664 4605 33554432000 134512640 135204385 4287367552 18446744073709551615 134543699 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/22511/statm: 69059 4606 57 169 0 68889 0
[pid=22512] ppid=22491 vsize=440936 CPUtime=0 cores=0-7
/proc/22512/stat : 22512 (python) S 22491 22491 21696 0 -1 4202560 761 0 0 0 0 0 0 0 18 0 1 0 548555550 451518464 16952 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 14214346415328460799 0 0 17 1 0 0 0
/proc/22512/statm: 110234 16952 347 364 0 46137 0
[pid=22514] ppid=22512 vsize=276236 CPUtime=1.43 cores=0-7
/proc/22514/stat : 22514 (hybridGM3) R 22512 22514 22514 0 -1 4202496 4846 0 0 0 143 0 0 0 25 0 1 0 548555550 282865664 4577 33554432000 134512640 135204385 4286793184 18446744073709551615 134543830 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22514/statm: 69059 4577 57 169 0 68889 0
[pid=22515] ppid=22491 vsize=451192 CPUtime=0 cores=0-7
/proc/22515/stat : 22515 (python) S 22491 22491 21696 0 -1 4202560 731 0 0 0 0 0 0 0 18 0 1 0 548555554 462020608 17129 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 14214357960200552447 0 0 17 1 0 0 0
/proc/22515/statm: 112798 17129 348 364 0 48698 0
[pid=22517] ppid=22515 vsize=276236 CPUtime=1.37 cores=0-7
/proc/22517/stat : 22517 (hybridGM3) R 22515 22517 22517 0 -1 4202496 4793 0 0 0 137 0 0 0 25 0 1 0 548555555 282865664 4538 33554432000 134512640 135204385 4294712912 18446744073709551615 134543993 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22517/statm: 69059 4538 57 169 0 68889 0
[pid=22518] ppid=22491 vsize=461448 CPUtime=0 cores=0-7
/proc/22518/stat : 22518 (python) S 22491 22491 21696 0 -1 4202560 770 0 0 0 0 0 0 0 18 0 1 0 548555558 472522752 17111 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 5723613335799201791 0 0 17 0 0 0 0
/proc/22518/statm: 115362 17111 354 364 0 51259 0
[pid=22520] ppid=22518 vsize=471116 CPUtime=1.35 cores=0-7
/proc/22520/stat : 22520 (TNM) R 22518 22520 22520 0 -1 4202496 8432 0 0 0 135 0 0 0 25 0 1 0 548555559 482422784 8171 33554432000 134512640 135008748 4294822192 18446744073709551615 134528483 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/22520/statm: 117779 8171 46 122 0 116679 0
[pid=22491/tid=22498] ppid=22489 vsize=471692 CPUtime=0 cores=0-7
/proc/22491/task/22498/stat : 22498 (python) S 22489 22491 21696 0 -1 4202560 27 934 0 0 0 0 0 0 18 0 9 0 548555530 483012608 18650 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 1 0 0 0
[pid=22491/tid=22501] ppid=22489 vsize=471692 CPUtime=0 cores=0-7
/proc/22491/task/22501/stat : 22501 (python) S 22489 22491 21696 0 -1 4202560 21 934 0 0 0 0 0 0 25 0 9 0 548555534 483012608 18650 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 0 0 0 0
[pid=22491/tid=22504] ppid=22489 vsize=471692 CPUtime=0 cores=0-7
/proc/22491/task/22504/stat : 22504 (python) S 22489 22491 21696 0 -1 4202560 23 934 0 0 0 0 0 0 19 0 9 0 548555538 483012608 18650 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 0 0 0 0
[pid=22491/tid=22507] ppid=22489 vsize=471692 CPUtime=0 cores=0-7
/proc/22491/task/22507/stat : 22507 (python) S 22489 22491 21696 0 -1 4202560 19 934 0 0 0 0 0 0 18 0 9 0 548555542 483012608 18650 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 5 0 0 0
[pid=22491/tid=22510] ppid=22489 vsize=471692 CPUtime=0 cores=0-7
/proc/22491/task/22510/stat : 22510 (python) S 22489 22491 21696 0 -1 4202560 28 934 0 0 0 0 0 0 19 0 9 0 548555546 483012608 18650 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 6 0 0 0
[pid=22491/tid=22513] ppid=22489 vsize=471692 CPUtime=0 cores=0-7
/proc/22491/task/22513/stat : 22513 (python) S 22489 22491 21696 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 9 0 548555550 483012608 18650 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 7 0 0 0
[pid=22491/tid=22516] ppid=22489 vsize=471692 CPUtime=0 cores=0-7
/proc/22491/task/22516/stat : 22516 (python) S 22489 22491 21696 0 -1 4202560 100 934 0 0 0 0 0 0 18 0 9 0 548555554 483012608 18650 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 1 0 0 0
[pid=22491/tid=22519] ppid=22489 vsize=471692 CPUtime=0 cores=0-7
/proc/22491/task/22519/stat : 22519 (python) S 22489 22491 21696 0 -1 4202560 33 934 0 0 0 0 0 0 20 0 9 0 548555559 483012608 18650 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 23.02
Current children cumulated vsize (KiB) 6018728

[startup+25.5015 s]
/proc/loadavg: 6.80 7.64 6.75 9/306 22520
/proc/meminfo: memFree=22953516/32950928 swapFree=54880832/67111528
[pid=22491] ppid=22489 vsize=471692 CPUtime=11.06 cores=0-7
/proc/22491/stat : 22491 (python) S 22489 22491 21696 0 -1 4202496 34005 934 1 0 1090 16 0 0 20 0 9 0 548554426 483012608 18650 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512518603 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/22491/statm: 117923 18650 1889 364 0 53820 0
[pid=22497] ppid=22491 vsize=324120 CPUtime=0.02 cores=0-7
/proc/22497/stat : 22497 (python) S 22491 22491 21696 0 -1 4202560 908 0 0 0 1 1 0 0 15 0 1 0 548555530 331898880 16959 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500711951 0 0 16781312 514 12345386925413073155 0 0 17 6 0 0 0
/proc/22497/statm: 81030 16959 376 364 0 16948 0
[pid=22499] ppid=22497 vsize=276236 CPUtime=14.45 cores=0-7
/proc/22499/stat : 22499 (hybridGM3) R 22497 22499 22499 0 -1 4202496 9521 0 0 0 1444 1 0 0 25 0 1 0 548555530 282865664 9254 33554432000 134512640 135204385 4288671952 18446744073709551615 134537403 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22499/statm: 69059 9254 64 169 0 68889 0
[pid=22500] ppid=22491 vsize=334376 CPUtime=0.02 cores=0-7
/proc/22500/stat : 22500 (python) S 22491 22491 21696 0 -1 4202560 905 0 0 0 1 1 0 0 15 0 1 0 548555534 342401024 16957 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 12345386925413073157 0 0 17 0 0 0 0
/proc/22500/statm: 83594 16957 370 364 0 19509 0
[pid=22502] ppid=22500 vsize=276236 CPUtime=14.4 cores=0-7
/proc/22502/stat : 22502 (hybridGM3) R 22500 22502 22502 0 -1 4202496 9521 0 0 0 1439 1 0 0 25 0 1 0 548555534 282865664 9255 33554432000 134512640 135204385 4288556448 18446744073709551615 134547471 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/22502/statm: 69059 9255 64 169 0 68889 0
[pid=22503] ppid=22491 vsize=344632 CPUtime=0.02 cores=0-7
/proc/22503/stat : 22503 (python) S 22491 22491 21696 0 -1 4202560 932 0 0 0 1 1 0 0 15 0 1 0 548555538 352903168 16961 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 12345386925413073153 0 0 17 1 0 0 0
/proc/22503/statm: 86158 16961 370 364 0 22070 0
[pid=22505] ppid=22503 vsize=276244 CPUtime=14.36 cores=0-7
/proc/22505/stat : 22505 (hybridGM3) R 22503 22505 22505 0 -1 4202496 9512 0 0 0 1435 1 0 0 25 0 1 0 548555538 282873856 9251 33554432000 134512640 135204385 4288886592 18446744073709551615 134536922 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/22505/statm: 69061 9251 64 169 0 68891 0
[pid=22506] ppid=22491 vsize=354888 CPUtime=0.02 cores=0-7
/proc/22506/stat : 22506 (python) S 22491 22491 21696 0 -1 4202560 951 0 0 0 1 1 0 0 15 0 1 0 548555542 363405312 16965 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 12345386925413073155 0 0 17 1 0 0 0
/proc/22506/statm: 88722 16965 370 364 0 24631 0
[pid=22508] ppid=22506 vsize=276240 CPUtime=14.33 cores=0-7
/proc/22508/stat : 22508 (hybridGM3) R 22506 22508 22508 0 -1 4202496 9514 0 0 0 1432 1 0 0 25 0 1 0 548555542 282869760 9251 33554432000 134512640 135204385 4289990240 18446744073709551615 134545864 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22508/statm: 69060 9251 64 169 0 68890 0
[pid=22509] ppid=22491 vsize=430680 CPUtime=0.02 cores=0-7
/proc/22509/stat : 22509 (python) S 22491 22491 21696 0 -1 4202560 952 0 0 0 1 1 0 0 15 0 1 0 548555546 441016320 16971 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 12345386925413073156 0 0 17 1 0 0 0
/proc/22509/statm: 107670 16971 370 364 0 43576 0
[pid=22511] ppid=22509 vsize=276240 CPUtime=14.25 cores=0-7
/proc/22511/stat : 22511 (hybridGM3) R 22509 22511 22511 0 -1 4202496 9515 0 0 0 1424 1 0 0 25 0 1 0 548555546 282869760 9250 33554432000 134512640 135204385 4287367552 18446744073709551615 134545968 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/22511/statm: 69060 9250 64 169 0 68890 0

################
# More data... #
################

/proc/22614/task/22636/stat : 22636 (python) S 22491 22491 21696 0 -1 4202560 81 0 0 0 0 0 0 0 19 0 2 0 548644205 630054912 17465 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 514 18446604471159088960 0 0 -1 2 0 0 0
[pid=22618] ppid=22491 vsize=625544 CPUtime=0.5 cores=0-7
/proc/22618/stat : 22618 (python) S 22491 22491 21696 0 -1 4202560 1195 0 0 0 26 24 0 0 15 0 2 0 548634603 640557056 17547 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/22618/statm: 156386 17547 406 364 0 92235 0
[pid=22621] ppid=22618 vsize=276240 CPUtime=201.97 cores=0-7
/proc/22621/stat : 22621 (hybridGM3) R 22618 22621 22621 0 -1 4202496 9544 0 0 0 20196 1 0 0 25 0 1 0 548634605 282869760 9256 33554432000 134512640 135204385 4286794528 18446744073709551615 134536641 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22621/statm: 69060 9256 64 169 0 68890 0
[pid=22618/tid=22657] ppid=22491 vsize=625544 CPUtime=0 cores=0-7
/proc/22618/task/22657/stat : 22657 (python) S 22491 22491 21696 0 -1 4202560 83 0 0 0 0 0 0 0 17 0 2 0 548654281 640557056 17547 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 514 18446604466224677120 0 0 -1 3 0 0 0
[pid=22623] ppid=22491 vsize=635800 CPUtime=0.25 cores=0-7
/proc/22623/stat : 22623 (python) S 22491 22491 21696 0 -1 4202560 1177 0 0 0 15 10 0 0 18 0 2 0 548634674 651059200 17502 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/22623/statm: 158950 17502 400 364 0 94796 0
[pid=22625] ppid=22623 vsize=471116 CPUtime=100.07 cores=0-7
/proc/22625/stat : 22625 (TNM) T 22623 22625 22625 0 -1 4202496 8455 0 0 0 10006 1 0 0 25 0 1 0 548634674 482422784 8172 33554432000 134512640 135008748 4292408768 18446744073709551615 134527267 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/22625/statm: 117779 8172 46 122 0 116679 0
[pid=22623/tid=22647] ppid=22491 vsize=635800 CPUtime=0 cores=0-7
/proc/22623/task/22647/stat : 22647 (python) S 22491 22491 21696 0 -1 4202560 88 0 0 0 0 0 0 0 19 0 2 0 548644709 651059200 17502 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 514 18446604462103553152 0 0 -1 2 0 0 0
[pid=22626] ppid=22491 vsize=646056 CPUtime=0.25 cores=0-7
/proc/22626/stat : 22626 (python) S 22491 22491 21696 0 -1 4202560 1180 0 0 0 15 10 0 0 19 0 2 0 548634684 661561344 17476 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/22626/statm: 161514 17476 400 364 0 97357 0
[pid=22628] ppid=22626 vsize=471116 CPUtime=98.76 cores=0-7
/proc/22628/stat : 22628 (TNM) T 22626 22628 22628 0 -1 4202496 8456 0 0 0 9874 2 0 0 25 0 1 0 548634685 482422784 8173 33554432000 134512640 135008748 4293104192 18446744073709551615 134527118 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/22628/statm: 117779 8173 46 122 0 116679 0
[pid=22626/tid=22643] ppid=22491 vsize=646056 CPUtime=0 cores=0-7
/proc/22626/task/22643/stat : 22643 (python) S 22491 22491 21696 0 -1 4202560 84 0 0 0 0 0 0 0 19 0 2 0 548644612 661561344 17476 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 514 18446604456036026240 0 0 -1 0 0 0 0
[pid=22630] ppid=22491 vsize=656312 CPUtime=0.25 cores=0-7
/proc/22630/stat : 22630 (python) S 22491 22491 21696 0 -1 4202560 1193 0 0 0 13 12 0 0 19 0 2 0 548634996 672063488 17617 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/22630/statm: 164078 17617 400 364 0 99918 0
[pid=22632] ppid=22630 vsize=471116 CPUtime=99 cores=0-7
/proc/22632/stat : 22632 (TNM) T 22630 22632 22632 0 -1 4202496 8453 0 0 0 9899 1 0 0 25 0 1 0 548634997 482422784 8171 33554432000 134512640 135008748 4292613696 18446744073709551615 134529176 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/22632/statm: 117779 8171 46 122 0 116679 0
[pid=22630/tid=22651] ppid=22491 vsize=656312 CPUtime=0 cores=0-7
/proc/22630/task/22651/stat : 22651 (python) S 22491 22491 21696 0 -1 4202560 89 0 0 0 0 0 0 0 18 0 2 0 548644925 672063488 17617 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 514 18446604449819265856 0 0 -1 1 0 0 0
[pid=22637] ppid=22491 vsize=666568 CPUtime=0.28 cores=0-7
/proc/22637/stat : 22637 (python) S 22491 22491 21696 0 -1 4202560 1011 0 0 0 14 14 0 0 18 0 1 0 548644215 682565632 17460 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 12345386925413073155 0 0 17 7 0 0 0
/proc/22637/statm: 166642 17460 373 364 0 102479 0
[pid=22639] ppid=22637 vsize=438604 CPUtime=106.32 cores=0-7
/proc/22639/stat : 22639 (adaptg2wsat2009) R 22637 22639 22639 0 -1 4202496 8788 0 0 0 10631 1 0 0 25 0 1 0 548644216 449130496 8497 33554432000 134512640 135107411 4293653632 18446744073709551615 134531246 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22639/statm: 109651 8497 59 146 0 108527 0
[pid=22640] ppid=22491 vsize=676824 CPUtime=0.25 cores=0-7
/proc/22640/stat : 22640 (python) S 22491 22491 21696 0 -1 4202560 1233 0 0 0 13 12 0 0 19 0 2 0 548644349 693067776 17475 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/22640/statm: 169206 17475 406 364 0 105040 0
[pid=22642] ppid=22640 vsize=471116 CPUtime=99.72 cores=0-7
/proc/22642/stat : 22642 (TNM) T 22640 22642 22642 0 -1 4202496 8460 0 0 0 9969 3 0 0 25 0 1 0 548644349 482422784 8170 33554432000 134512640 135008748 4287855904 18446744073709551615 134528967 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/22642/statm: 117779 8170 46 122 0 116679 0
[pid=22640/tid=22658] ppid=22491 vsize=676824 CPUtime=0 cores=0-7
/proc/22640/task/22658/stat : 22658 (python) S 22491 22491 21696 0 -1 4202560 90 0 0 0 0 0 0 0 19 0 2 0 548654373 693067776 17475 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 514 18446604449819264064 0 0 -1 4 0 0 0
[pid=22644] ppid=22491 vsize=687080 CPUtime=0.25 cores=0-7
/proc/22644/stat : 22644 (python) S 22491 22491 21696 0 -1 4202560 1242 0 0 0 16 9 0 0 19 0 2 0 548644624 703569920 17550 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/22644/statm: 171770 17550 407 364 0 107601 0
[pid=22646] ppid=22644 vsize=471116 CPUtime=100.07 cores=0-7
/proc/22646/stat : 22646 (TNM) T 22644 22646 22646 0 -1 4202496 8463 0 0 0 10006 1 0 0 25 0 1 0 548644625 482422784 8172 33554432000 134512640 135008748 4289557568 18446744073709551615 134529300 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/22646/statm: 117779 8172 46 122 0 116679 0
[pid=22644/tid=22662] ppid=22491 vsize=687080 CPUtime=0 cores=0-7
/proc/22644/task/22662/stat : 22662 (python) S 22491 22491 21696 0 -1 4202560 89 0 0 0 0 0 0 0 19 0 2 0 548654659 703569920 17550 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 514 18446604453906375808 0 0 -1 2 0 0 0
[pid=22648] ppid=22491 vsize=697336 CPUtime=0.25 cores=0-7
/proc/22648/stat : 22648 (python) S 22491 22491 21696 0 -1 4202560 1253 0 0 0 12 13 0 0 19 0 2 0 548644720 714072064 17614 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512518603 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/22648/statm: 174334 17614 407 364 0 110162 0
[pid=22650] ppid=22648 vsize=471116 CPUtime=100.23 cores=0-7
/proc/22650/stat : 22650 (TNM) T 22648 22650 22650 0 -1 4202496 8462 0 0 0 10022 1 0 0 25 0 1 0 548644721 482422784 8172 33554432000 134512640 135008748 4294581760 18446744073709551615 134524687 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/22650/statm: 117779 8172 46 122 0 116679 0
[pid=22648/tid=22663] ppid=22491 vsize=697336 CPUtime=0 cores=0-7
/proc/22648/task/22663/stat : 22663 (python) S 22491 22491 21696 0 -1 4202560 92 0 0 0 0 0 0 0 19 0 2 0 548654745 714072064 17614 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 514 18446604450888837120 0 0 -1 1 0 0 0
[pid=22652] ppid=22491 vsize=707592 CPUtime=0.24 cores=0-7
/proc/22652/stat : 22652 (python) S 22491 22491 21696 0 -1 4202560 1058 0 0 0 12 12 0 0 18 0 1 0 548644937 724574208 17583 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 12345386925413073155 0 0 17 4 0 0 0
/proc/22652/statm: 176898 17583 379 364 0 112723 0
[pid=22654] ppid=22652 vsize=471116 CPUtime=97.85 cores=0-7
/proc/22654/stat : 22654 (TNM) R 22652 22654 22654 0 -1 4202496 8461 0 0 0 9783 2 0 0 25 0 1 0 548644938 482422784 8170 33554432000 134512640 135008748 4292790512 18446744073709551615 134527339 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/22654/statm: 117779 8170 46 122 0 116679 0
[pid=22659] ppid=22491 vsize=717848 CPUtime=0.02 cores=0-7
/proc/22659/stat : 22659 (python) S 22491 22491 21696 0 -1 4202560 1051 0 0 0 1 1 0 0 18 0 1 0 548654385 735076352 17520 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 12991690309875237123 0 0 17 4 0 0 0
/proc/22659/statm: 179462 17520 372 364 0 115284 0
[pid=22661] ppid=22659 vsize=471116 CPUtime=4.6 cores=0-7
/proc/22661/stat : 22661 (TNM) R 22659 22661 22661 0 -1 4202496 8468 0 0 0 459 1 0 0 25 0 1 0 548654386 482422784 8172 33554432000 134512640 135008748 4291045360 18446744073709551615 134527840 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/22661/statm: 117779 8172 46 122 0 116679 0
[pid=22664] ppid=22491 vsize=728104 CPUtime=0 cores=0-7
/proc/22664/stat : 22664 (python) S 22491 22491 21696 0 -1 4202560 923 0 0 0 0 0 0 0 19 0 1 0 548654757 745578496 17695 33554432000 4194304 5683644 140735917409424 18446744073709551615 254500712006 0 0 16781312 514 9647418491610857471 0 0 17 4 0 0 0
/proc/22664/statm: 182026 17695 346 364 0 117845 0
[pid=22666] ppid=22664 vsize=471116 CPUtime=0.88 cores=0-7
/proc/22666/stat : 22666 (TNM) R 22664 22666 22666 0 -1 4202496 7238 0 0 0 87 1 0 0 23 0 1 0 548654757 482422784 6950 33554432000 134512640 135008748 4291792656 18446744073709551615 134520494 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/22666/statm: 117779 6950 46 122 0 116679 0
[pid=22491/tid=22498] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22498/stat : 22498 (python) S 22489 22491 21696 0 -1 4202560 35 934 0 0 0 0 0 0 15 0 35 0 548555530 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 2 0 0 0
[pid=22491/tid=22501] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22501/stat : 22501 (python) S 22489 22491 21696 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 35 0 548555534 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 4 0 0 0
[pid=22491/tid=22504] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22504/stat : 22504 (python) S 22489 22491 21696 0 -1 4202560 44 934 0 0 0 0 0 0 18 0 35 0 548555538 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 0 0 0 0
[pid=22491/tid=22507] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22507/stat : 22507 (python) S 22489 22491 21696 0 -1 4202560 27 934 0 0 0 0 0 0 18 0 35 0 548555542 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 5 0 0 0
[pid=22491/tid=22510] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22510/stat : 22510 (python) S 22489 22491 21696 0 -1 4202560 34 934 0 0 0 0 0 0 18 0 35 0 548555546 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 6 0 0 0
[pid=22491/tid=22513] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22513/stat : 22513 (python) S 22489 22491 21696 0 -1 4202560 45 934 0 0 0 0 0 0 18 0 35 0 548555550 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 7 0 0 0
[pid=22491/tid=22516] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22516/stat : 22516 (python) S 22489 22491 21696 0 -1 4202560 106 934 0 0 0 0 0 0 18 0 35 0 548555554 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 1 0 0 0
[pid=22491/tid=22519] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22519/stat : 22519 (python) S 22489 22491 21696 0 -1 4202560 40 934 0 0 0 0 0 0 18 0 35 0 548555559 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 0 0 0 0
[pid=22491/tid=22529] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22529/stat : 22529 (python) S 22489 22491 21696 0 -1 4202560 57 934 0 0 0 0 0 0 18 0 35 0 548565584 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 2 0 0 0
[pid=22491/tid=22541] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22541/stat : 22541 (python) S 22489 22491 21696 0 -1 4202560 61 934 0 0 0 0 0 0 18 0 35 0 548584721 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 1 0 0 0
[pid=22491/tid=22544] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22544/stat : 22544 (python) S 22489 22491 21696 0 -1 4202560 50 934 0 0 0 0 0 0 18 0 35 0 548585220 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 4 0 0 0
[pid=22491/tid=22547] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22547/stat : 22547 (python) S 22489 22491 21696 0 -1 4202560 37 934 0 0 0 0 0 0 18 0 35 0 548585225 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 0 0 0 0
[pid=22491/tid=22551] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22551/stat : 22551 (python) S 22489 22491 21696 0 -1 4202560 36 934 0 0 0 0 0 0 21 0 35 0 548585255 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 2 0 0 0
[pid=22491/tid=22556] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22556/stat : 22556 (python) S 22489 22491 21696 0 -1 4202560 65 934 0 0 0 0 0 0 15 0 35 0 548594757 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 1 0 0 0
[pid=22491/tid=22559] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22559/stat : 22559 (python) S 22489 22491 21696 0 -1 4202560 39 934 0 0 0 0 0 0 18 0 35 0 548594862 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 2 0 0 0
[pid=22491/tid=22564] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22564/stat : 22564 (python) S 22489 22491 21696 0 -1 4202560 38 934 0 0 0 0 0 0 18 0 35 0 548595294 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 2 0 0 0
[pid=22491/tid=22585] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22585/stat : 22585 (python) S 22489 22491 21696 0 -1 4202560 47 934 0 0 0 0 0 0 19 0 35 0 548614533 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 2 0 0 0
[pid=22491/tid=22589] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22589/stat : 22589 (python) S 22489 22491 21696 0 -1 4202560 40 934 0 0 0 0 0 0 20 0 35 0 548615023 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 2 0 0 0
[pid=22491/tid=22599] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22599/stat : 22599 (python) S 22489 22491 21696 0 -1 4202560 39 934 0 0 0 0 0 0 19 0 35 0 548624564 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 2 0 0 0
[pid=22491/tid=22602] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22602/stat : 22602 (python) S 22489 22491 21696 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 35 0 548624643 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 1 0 0 0
[pid=22491/tid=22605] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22605/stat : 22605 (python) S 22489 22491 21696 0 -1 4202560 36 934 0 0 0 0 0 0 20 0 35 0 548624674 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 1 0 0 0
[pid=22491/tid=22609] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22609/stat : 22609 (python) S 22489 22491 21696 0 -1 4202560 32 934 0 0 0 0 0 0 19 0 35 0 548624960 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 0 0 0 0
[pid=22491/tid=22615] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22615/stat : 22615 (python) S 22489 22491 21696 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 35 0 548634180 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 0 0 0 0
[pid=22491/tid=22619] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22619/stat : 22619 (python) S 22489 22491 21696 0 -1 4202560 37 934 0 0 0 0 0 0 18 0 35 0 548634603 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 1 0 0 0
[pid=22491/tid=22624] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22624/stat : 22624 (python) S 22489 22491 21696 0 -1 4202560 34 934 0 0 0 0 0 0 20 0 35 0 548634674 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 1 0 0 0
[pid=22491/tid=22627] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22627/stat : 22627 (python) S 22489 22491 21696 0 -1 4202560 33 934 0 0 0 0 0 0 19 0 35 0 548634684 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 7 0 0 0
[pid=22491/tid=22631] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22631/stat : 22631 (python) S 22489 22491 21696 0 -1 4202560 34 934 0 0 0 0 0 0 20 0 35 0 548634997 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 1 0 0 0
[pid=22491/tid=22638] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22638/stat : 22638 (python) S 22489 22491 21696 0 -1 4202560 32 934 0 0 0 0 0 0 20 0 35 0 548644215 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 1 0 0 0
[pid=22491/tid=22641] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22641/stat : 22641 (python) S 22489 22491 21696 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 35 0 548644349 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 4 0 0 0
[pid=22491/tid=22645] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22645/stat : 22645 (python) S 22489 22491 21696 0 -1 4202560 32 934 0 0 0 0 0 0 20 0 35 0 548644625 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 4 0 0 0
[pid=22491/tid=22649] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22649/stat : 22649 (python) S 22489 22491 21696 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 35 0 548644720 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 4 0 0 0
[pid=22491/tid=22653] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22653/stat : 22653 (python) S 22489 22491 21696 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 35 0 548644938 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 4 0 0 0
[pid=22491/tid=22660] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22660/stat : 22660 (python) S 22489 22491 21696 0 -1 4202560 35 934 0 0 0 0 0 0 19 0 35 0 548654385 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 4 0 0 0
[pid=22491/tid=22665] ppid=22489 vsize=738348 CPUtime=0 cores=0-7
/proc/22491/task/22665/stat : 22665 (python) S 22489 22491 21696 0 -1 4202560 33 934 0 0 0 0 0 0 20 0 35 0 548654757 756068352 19295 33554432000 4194304 5683644 140735917409424 18446744073709551615 254512515505 0 0 16781312 2 18446604443337328576 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 7924.96
Current children cumulated vsize (KiB) 32778544

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 22491 and gives
#  childrusage.ru_utime.tv_sec=1711
#  childrusage.ru_utime.tv_usec=681784
#  childrusage.ru_stime.tv_sec=9
#  childrusage.ru_stime.tv_usec=455562
# CPU time returned by wait4() is 1721.14
# while last known CPU time is 7924.96
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1004.41
CPU time (s): 7924.96
CPU user time (s): 7914.99
CPU system time (s): 9.97001
CPU usage (%): 789.013
Max. virtual memory (cumulated for all children) (KiB): 32778544

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1711.68
system time used= 9.45556
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 155160
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 12435
involuntary context switches= 18799

runsolver used 7.80581 second user time and 20.3459 second system time

The end

Launcher Data 

Begin job on node149 at 2011-05-02 22:52:57
IDJOB=3321923
IDBENCH=82701
IDSOLVER=1712
FILE ID=node149/3321923-1304369577
RUNJOBID= node149-1304366464-21710
PBS_JOBID= 13181725
Free space on /tmp= 73540 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v50000-c210000-S964643921-035-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321923-1304369577/watcher-3321923-1304369577 -o /tmp/evaluation-result-3321923-1304369577/solver-3321923-1304369577 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321923-1304369577.cnf -seed 992975932 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= d8337de8a71a630ccf2dd02943cd7d7f
RANDOM SEED=992975932

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      23343516 kB
Buffers:         11084 kB
Cached:         192240 kB
SwapCached:      27552 kB
Active:         407616 kB
Inactive:      9017436 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      23343516 kB
SwapTotal:    67111528 kB
SwapFree:     54880832 kB
Dirty:           36132 kB
Writeback:           0 kB
AnonPages:     9205828 kB
Mapped:          13088 kB
Slab:            57020 kB
PageTables:      58004 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 55875316 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73512 MiB
End job on node149 at 2011-05-02 23:09:44