Trace number 3293702

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
CryptoMiniSat TechDemo-Alpha-mt? (TO) 4800.32 1206.19

General information on the benchmark

NameSAT-Race-2010-CNF/hardware-verification/
velev/11pipe_k.cnf
MD5SUM932387240a0c17f4c88af52c4ca151c6
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark325.87
Satisfiable
(Un)Satisfiability was proved
Number of variables89315
Number of clauses5584003
Sum of the clauses size16593705
Maximum clause length295
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 25473787
Number of clauses of size 337063
Number of clauses of size 4514
Number of clauses of size 5453
Number of clauses of size over 572185

Solver Data 

0.00/0.00	c Outputting solution to console
0.00/0.00	c This is CryptoMiniSat 2.9.0
0.00/0.00	c compiled with gcc version 4.4.5
0.00/0.00	c Using 4 thread(s)
0.00/0.00	c WARNING: for repeatability, setting FPU to use double precision
0.00/0.00	c Reading file 'HOME/instance-3293702-1304182953.cnf'
0.00/0.00	c -- header says num vars:          89315
0.00/0.00	c -- header says num clauses:     5584003
5.99/6.02	c -- clauses added:            0 learnts,      5584003 normals,            0 xors
5.99/6.02	c -- vars added      89315
5.99/6.02	c Parsing time:  5.77 s
5.99/6.03	c  N st     0         0     89315    110215   5473787         0  16593704         0   no data   no data  --
7.92/6.52	c asymm  cl-useful: 14/15/110213 lits-rem:240 time: 1.88
27.47/11.40	c Flit:    99 Blit:      0 bXBeca:    0 bXProp:    0 Bin:      0 BRemL:      0 BRemN:      0 CBin:       0 P: 60.0M T: 19.18
29.44/11.99	c Calc non-exist non-lernt bins v-fix:     0 done:  28466 time:  2.18 s
29.44/11.99	c Cache  BProp: 0 Set: 0 BXProp: 0 T: 0.00
29.83/12.06	c Subs w/ non-existent bins:      0 time:  0.22 s
30.99/12.36	c Replacing        2 vars Replaced      381 lits Time:     1.19 s 
36.19/13.66	c bin-w-bin subsume rem          165 bins  time:  4.23 s
43.40/15.42	c subs with bin:        0  lits-rem:         6  v-fix:    0  time:  7.02 s
44.18/15.67	c Removed useless bin:       0  fixed:     0  bprops:  10.00M  time:  0.98 s
206.35/56.21	c lits-rem:         0  cl-subs:    14325  v-elim:   7346  v-fix:    0  time: 161.89 s
206.35/56.21	c learnt bin added due to v-elim: 0
206.35/56.27	c Finding binary XORs  T:     0.25 s  found:      12
207.15/56.45	c Replacing       12 vars Replaced      120 lits Time:     0.72 s 
208.75/56.84	c Finding non-binary XORs:     0.82 s (found:       0, avg size: nan)
208.75/56.84	c Calc default polars -  time:   0.26 s pos:    5381 undec:    7417 neg:   76517
208.75/56.84	c =========================================================================================
208.75/56.84	c types(t): F = full restart, N = normal restart
208.75/56.84	c types(t): S = simplification begin/end, E = solution found
208.75/56.84	c restart types(rt): st = static, dy = dynamic
208.75/56.84	c  t rt  Rest     Confl      Vars   NormCls    BinCls   Learnts    ClLits    LtLits   MTavgCS    LTAvgG
208.75/56.84	c  B st     0         0     82117     84813   5459475         0  14801521         0   no data   no data  --
214.36/58.22	c Decided on dynamic restart strategy
219.15/59.42	c  N dy    19      4016     81981     84813   5459555      3771  14801521    491899    153.40     40.98  --
227.94/61.63	c  N dy    46      8097     81981     84813   5459616      7625  14801520   1011721    116.73     48.39  --
235.51/63.51	c  N dy    72     12105     81981     84813   5459666     11448  14801520   1725812    171.24     56.05  --
243.10/65.41	c  N dy    99     16277     81980     84813   5459971     15374  14801511   2270896    141.17     57.81  --
249.91/67.18	c  N dy   125     20356     81980     84813   5460008      9785  14801511    669088    282.61     56.84  --
255.88/68.68	c  N dy   151     24497     81980     84813   5460058     13663  14801508   1314136    113.37     57.55  --
260.67/69.86	c  N dy   177     28669     81980     84813   5460083     17631  14801503   1725225    149.63     57.25  --
263.86/70.66	c  S st   186     30002     81980     84813   5460092     18883  14801503   1826921     83.90     56.87  --
268.66/71.88	c  S st   186     30502     81976     84813   5460234     19236  14801503   1837679     83.90     56.87  --
269.46/72.02	c x-sub:     0 x-cut:      0 vfix:      0 v-elim:      0 locsubst:     0 time:   0.56
1664.04/421.18	c Flit:  1058 Blit:      2 bXBeca:    0 bXProp:    0 Bin:  47120 BRemL:      0 BRemN:     62 CBin:       0 P: 60.0M T: 1394.79
1664.45/421.29	c Cleaning up after failed var search:     0.43 s 
1667.22/421.98	c Calc non-exist non-lernt bins v-fix:     0 done:  26354 time:  2.70 s
1667.22/421.98	c Cache  BProp: 0 Set: 0 BXProp: 1 T: 0.00
1667.62/422.05	c Subs w/ non-existent bins:      0 time:  0.29 s
1668.42/422.23	c Replacing        1 vars Replaced      340 lits Time:     0.68 s 
1673.23/423.50	c bin-w-bin subsume rem           18 bins  time:  4.26 s
1678.81/424.83	c subs with bin:      152  lits-rem:     11218  v-fix:    0  time:  5.32 s
1679.60/425.08	c Removed useless bin:     134  fixed:     0  bprops:  10.03M  time:  1.01 s
1707.57/432.05	c lits-rem:       449  cl-subs:     9933  v-elim:  31756  v-fix:    0  time: 27.79 s
1707.57/432.05	c learnt bin added due to v-elim: 0
1708.79/432.32	c asymm  cl-useful: 28/31/135097 lits-rem:251 time: 1.08
1709.16/432.45	c  N dy   186     30502     78000    135097   5297899      9202  15428093    694271     83.90     56.87  --
1709.16/432.45	c  F st   186     30502     78000    135097   5297899      9202  15428093    694271     83.90     56.87  --
1714.75/433.83	c Decided on dynamic restart strategy
1718.35/434.72	c  N dy   205     34554     78000    135097   5297911     13051  15428093   1174744    139.83     59.21  --
1725.54/436.56	c  N dy   231     38651     78000    135097   5297954     16978  15428093   1671770    126.95     60.30  --
1732.33/438.24	c  N dy   257     42795     78000    135097   5297977     20828  15428093   2246629    160.81     60.20  --
1739.13/439.96	c  S st   277     45753     78000    135097   5298014     23652  15428092   2590260    118.11     60.48  --
1745.51/441.54	c  S st   277     46253     77995    135097   5298138     24009  15428091   2602342    118.11     60.48  --
1745.91/441.68	c x-sub:     0 x-cut:      0 vfix:      0 v-elim:      0 locsubst:     0 time:   0.57
2648.74/667.64	c Flit:   811 Blit:      2 bXBeca:    0 bXProp:    0 Bin:  34347 BRemL:     25 BRemN:      0 CBin:       0 P: 60.1M T: 902.58
2649.14/667.75	c Cleaning up after failed var search:     0.44 s 
2651.12/668.25	c Calc non-exist non-lernt bins v-fix:     0 done:  28523 time:  1.95 s
2651.12/668.25	c Cache  BProp: 0 Set: 0 BXProp: 0 T: 0.01
2651.52/668.36	c Subs w/ non-existent bins:    106 time:  0.46 s
2656.71/669.63	c bin-w-bin subsume rem          108 bins  time:  4.15 s
2661.91/670.97	c subs with bin:      177  lits-rem:     26838  v-fix:    0  time:  5.32 s
2662.70/671.19	c Removed useless bin:     323  fixed:     0  bprops:  10.00M  time:  0.91 s
2697.84/679.92	c lits-rem:      1983  cl-subs:     4115  v-elim:   3555  v-fix:    0  time: 34.84 s
2697.84/679.92	c learnt bin added due to v-elim: 0
2698.64/680.13	c asymm  cl-useful: 35/35/153516 lits-rem:276 time: 0.87
2699.04/680.25	c  N dy   277     46253     76951    153516   5204243     19907  15291562   2142062    118.11     60.48  --
2705.85/681.98	c  N dy   304     50366     76951    153516   5204267     23714  15291561   2685262    172.08     60.20  --
2712.63/683.64	c  N dy   331     54510     76951    153516   5204293     27621  15291561   3213959    127.73     60.14  --
2719.43/685.38	c  N dy   358     58624     76951    153516   5204338     17399  15291559   1191126    101.55     59.87  --
2725.81/686.92	c  N dy   384     62722     76951    153516   5204350     21274  15291558   1770070    143.81     59.44  --
2733.00/688.70	c  N dy   411     66792     76951    153516   5204383     25142  15291558   2232950    106.38     59.41  --
2739.39/690.33	c  S st   429     69379     76951    153516   5204395     27635  15291538   2477236     92.79     59.63  --
2744.18/691.53	c  S st   429     69879     76940    153516   5204522     27995  15291538   2494995     92.79     59.63  --
2744.58/691.66	c x-sub:     0 x-cut:      0 vfix:      0 v-elim:      0 locsubst:     0 time:   0.55
3784.57/951.97	c Flit:   440 Blit:      0 bXBeca:    0 bXProp:    0 Bin:  50785 BRemL:     13 BRemN:    105 CBin:       0 P: 60.1M T: 1039.90
3784.57/951.97	c Cleaning up after failed var search:     0.00 s 
3786.98/952.54	c Calc non-exist non-lernt bins v-fix:     0 done:  27333 time:  2.26 s
3786.98/952.54	c Cache  BProp: 0 Set: 0 BXProp: 0 T: 0.01
3794.15/954.38	c Subs w/ non-existent bins:      2 time:  7.34 s
3799.35/955.67	c bin-w-bin subsume rem          105 bins  time:  4.15 s
3806.54/957.47	c subs with bin:       98  lits-rem:     23644  v-fix:    0  time:  7.19 s
3808.14/957.84	c Removed useless bin:    3301  fixed:     0  bprops:  10.02M  time:  1.46 s
3875.65/974.72	c ORs :   8266 avg-s: 17.7 cl-sh: 49639 l-rem: 542900 b-add:      1 v-rep:   0 cl-rem: 627 avg s: 11.4 T:   33.85
3882.05/976.30	c Added 51 vars  tried: 80000 time: 6.32
3882.45/976.49	c ORs :     51 avg-s:  0.0 cl-sh: 32748 l-rem:  32748 b-add:    102 v-rep:   0 cl-rem: 24 avg s: 61.9 T:    7.08
3883.25/976.63	c lits-rem:     34382  cl-subs:     4856  v-elim:   3804  v-fix:    0  time: 75.06 s
3883.25/976.63	c learnt bin added due to v-elim: 0
3884.07/976.88	c asymm  cl-useful: 66/106/152565 lits-rem:3163 time: 0.99
3884.84/977.00	c  N dy   429     69879     76090    152557   5178325     23172  14658118   1954582     92.79     59.63  --
3884.84/977.00	c  F st   429     69879     76090    152557   5178325     23172  14658118   1954582     92.79     59.63  --
3890.83/978.56	c Decided on static restart strategy
3895.63/979.72	c  N dy   452     74458     73757    152557   5223589     33078  14658118   2867371    269.07     58.40  --
3905.22/982.12	c  N dy   485     79309     73739    152557   5223691     37471  14658117   4145485    272.31     57.78  --
3914.82/984.54	c  N dy   518     84370     73718    152557   5223732     22029  14658117   1656450    222.75     57.16  --
3924.40/986.93	c  N dy   555     89808     73706    152557   5223803     26976  14658114   2799938    215.25     56.58  --
3934.78/989.56	c  N dy   594     95562     73697    152557   5223843     32515  14658114   3962571    274.07     55.07  --
3945.59/992.22	c  N dy   635    101714     73690    152557   5223902     38139  14658113   5450286    255.19     54.57  --
3953.98/994.40	c  S st   657    104820     73687    152557   5223956     41001  14658113   6074951    252.42     54.06  --
3958.77/995.59	c  S st   657    105320     73687    152557   5224089     20796  14658113   1149568    252.42     54.06  --
3959.55/995.73	c x-sub:     0 x-cut:      0 vfix:      0 v-elim:      0 locsubst:     0 time:   0.54
4081.83/1026.35	c Flit:   192 Blit:      2 bXBeca:    0 bXProp:    0 Bin:   6939 BRemL:    222 BRemN:    241 CBin:       0 P: 60.0M T: 122.29
4082.21/1026.49	c Cleaning up after failed var search:     0.55 s 
4085.00/1027.16	c Calc non-exist non-lernt bins v-fix:     0 done:  26517 time:  2.68 s
4085.00/1027.16	c Cache  BProp: 7 Set: 7 BXProp: 1 T: 0.00
4093.40/1029.25	c Subs w/ non-existent bins:    318 time:  8.35 s
4094.61/1029.52	c Replacing        1 vars Replaced      327 lits Time:     1.07 s 
4099.01/1030.67	c bin-w-bin subsume rem          580 bins  time:  3.85 s
4104.58/1032.00	c subs with bin:      601  lits-rem:     39789  v-fix:    0  time:  5.32 s
4105.38/1032.24	c Removed useless bin:    3078  fixed:     0  bprops:  10.00M  time:  0.96 s
4128.96/1038.17	c ORs :   7872 avg-s: 13.3 cl-sh:  4497 l-rem:  17676 b-add:      5 v-rep:   0 cl-rem: 150 avg s: 10.6 T:    5.98
4129.77/1038.30	c lits-rem:      2367  cl-subs:     5572  v-elim:   5340  v-fix:    0  time: 24.20 s
4129.77/1038.30	c learnt bin added due to v-elim: 0
4130.17/1038.47	c asymm  cl-useful: 110/111/149873 lits-rem:1480 time: 0.68
4130.55/1038.58	c  N dy   657    105320     73305    149873   4871688     16209  13800980    926564    252.42     54.06  --
4144.14/1041.94	c  N dy   705    112176     73305    149873   4871777     22708  13800976   2689357    353.51     53.15  --
4158.12/1045.41	c  N dy   754    119424     73305    149873   4871839     29606  13800974   4641985    192.04     52.37  --
4171.71/1048.80	c  N dy   806    127076     73305    149873   4871877     36888  13800974   6466559    263.11     51.81  --
4190.49/1053.59	c  N dy   862    135185     73305    149873   4872037     23059  13800973   2013674    436.78     51.90  --
4206.46/1057.54	c  N dy   921    143886     73305    149873   4872124     31255  13800972   4543754    300.75     51.67  --
4225.24/1062.21	c  N dy   984    153160     73305    149873   4872210     40049  13800969   7215715    370.34     51.64  --
4234.84/1064.64	c  S st  1016    157982     73305    149873   4872236     44430  13800969   7827349    248.22     51.75  --
4239.65/1065.81	c  S st  1016    158482     73305    149873   4872372     22405  13800969   1428546    248.22     51.75  --
4240.06/1065.94	c x-sub:     0 x-cut:      0 vfix:      0 v-elim:      0 locsubst:     0 time:   0.52

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3293702-1304182953/watcher-3293702-1304182953 -o /tmp/evaluation-result-3293702-1304182953/solver-3293702-1304182953 -C 4800 -W 1300 -M 15500 ./techdemo --threads=4 HOME/instance-3293702-1304182953.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.61 5.61 5.94 6/180 12537
/proc/meminfo: memFree=29950232/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=50576 CPUtime=0 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 962 0 0 0 0 0 0 0 22 0 1 0 262486683 51789824 897 33554432000 4194304 4797429 140734255663168 18446744073709551615 268660323013 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/12537/statm: 12644 897 348 148 0 7573 0

[startup+0.020927 s]
/proc/loadavg: 7.61 5.61 5.94 6/180 12537
/proc/meminfo: memFree=29950232/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=53152 CPUtime=0.01 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 1580 0 0 0 1 0 0 0 22 0 1 0 262486683 54427648 1511 33554432000 4194304 4797429 140734255663168 18446744073709551615 4431562 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/12537/statm: 13326 1519 348 148 0 8255 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 53152

[startup+0.100913 s]
/proc/loadavg: 7.61 5.61 5.94 6/180 12537
/proc/meminfo: memFree=29950232/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=61904 CPUtime=0.09 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 3987 0 0 0 9 0 0 0 22 0 1 0 262486683 63389696 3884 33554432000 4194304 4797429 140734255663168 18446744073709551615 268660323013 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/12537/statm: 15476 3884 348 148 0 10405 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 61904

[startup+0.300878 s]
/proc/loadavg: 7.61 5.61 5.94 6/180 12537
/proc/meminfo: memFree=29950232/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=81932 CPUtime=0.29 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 9934 0 0 0 28 1 0 0 23 0 1 0 262486683 83898368 9412 33554432000 4194304 4797429 140734255663168 18446744073709551615 4365820 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/12537/statm: 20483 9412 348 148 0 15412 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 81932

[startup+0.700809 s]
/proc/loadavg: 7.61 5.61 5.94 6/180 12537
/proc/meminfo: memFree=29950232/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=120976 CPUtime=0.69 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 21303 0 0 0 67 2 0 0 25 0 1 0 262486683 123879424 19846 33554432000 4194304 4797429 140734255663168 18446744073709551615 268660323013 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/12537/statm: 30244 19866 349 148 0 25173 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 120976

[startup+1.50067 s]
/proc/loadavg: 7.40 5.60 5.93 6/181 12538
/proc/meminfo: memFree=29837756/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=202336 CPUtime=1.49 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 44538 0 0 0 143 6 0 0 25 0 1 0 262486683 207192064 41210 33554432000 4194304 4797429 140734255663168 18446744073709551615 268660323013 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/12537/statm: 50584 41210 349 148 0 45513 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 202336

[startup+3.10038 s]
/proc/loadavg: 7.40 5.60 5.93 6/181 12538
/proc/meminfo: memFree=29718844/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=465184 CPUtime=3.09 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 88233 0 0 0 296 13 0 0 25 0 1 0 262486683 476348416 81224 33554432000 4194304 4797429 140734255663168 18446744073709551615 268660323013 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/12537/statm: 116296 81224 349 148 0 111225 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 465184

[startup+6.30079 s]
/proc/loadavg: 7.40 5.60 5.93 6/181 12538
/proc/meminfo: memFree=29382652/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=986480 CPUtime=7.1 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 177966 0 0 0 684 26 0 0 18 0 4 0 262486683 1010155520 163402 33554432000 4194304 4797429 140734255663168 18446744073709551615 4428866 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12537/statm: 246620 163402 376 148 0 241549 0
Current children cumulated CPU time (s) 7.1
Current children cumulated vsize (KiB) 986480

[startup+12.7007 s]
/proc/loadavg: 7.27 5.63 5.94 9/184 12541
/proc/meminfo: memFree=29127076/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=1677608 CPUtime=32.59 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 277143 0 0 0 3207 52 0 0 18 0 4 0 262486683 1717870592 239604 33554432000 4194304 4797429 140734255663168 18446744073709551615 4557698 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12537/statm: 419402 239608 393 148 0 414331 0
[pid=12537/tid=12539] ppid=12535 vsize=1677608 CPUtime=6.63 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 25874 0 0 0 656 7 0 0 18 0 4 0 262487286 1717870592 239609 33554432000 4194304 4797429 140734255663168 18446744073709551615 268660323361 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=1677608 CPUtime=6.61 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 25945 0 0 0 656 5 0 0 18 0 4 0 262487286 1717870592 239612 33554432000 4194304 4797429 140734255663168 18446744073709551615 4557713 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=1677608 CPUtime=6.65 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 26096 0 0 0 657 8 0 0 18 0 4 0 262487286 1717870592 239618 33554432000 4194304 4797429 140734255663168 18446744073709551615 4501744 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 32.59
Current children cumulated vsize (KiB) 1677608

[startup+25.5004 s]
/proc/loadavg: 7.39 5.71 5.96 9/184 12541
/proc/meminfo: memFree=28913940/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=1779812 CPUtime=83.72 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 298422 0 0 0 8314 58 0 0 25 0 4 0 262486683 1822527488 260285 33554432000 4194304 4797429 140734255663168 18446744073709551615 4506580 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12537/statm: 444953 260285 537 148 0 439882 0
[pid=12537/tid=12539] ppid=12535 vsize=1779812 CPUtime=19.43 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 35116 0 0 0 1934 9 0 0 25 0 4 0 262487286 1822527488 260285 33554432000 4194304 4797429 140734255663168 18446744073709551615 4506622 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=1779812 CPUtime=19.34 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 28923 0 0 0 1927 7 0 0 25 0 4 0 262487286 1822527488 260285 33554432000 4194304 4797429 140734255663168 18446744073709551615 4506535 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=1779812 CPUtime=19.45 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 32621 0 0 0 1936 9 0 0 25 0 4 0 262487286 1822527488 260285 33554432000 4194304 4797429 140734255663168 18446744073709551615 4506622 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 83.72
Current children cumulated vsize (KiB) 1779812

[startup+51.1009 s]
/proc/loadavg: 7.59 5.89 6.01 9/184 12542
/proc/meminfo: memFree=28888496/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=1779812 CPUtime=185.98 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 298564 0 0 0 18540 58 0 0 25 0 4 0 262486683 1822527488 260294 33554432000 4194304 4797429 140734255663168 18446744073709551615 4506625 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12537/statm: 444953 260294 524 148 0 439882 0
[pid=12537/tid=12539] ppid=12535 vsize=1779812 CPUtime=45.03 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 35116 0 0 0 4494 9 0 0 25 0 4 0 262487286 1822527488 260294 33554432000 4194304 4797429 140734255663168 18446744073709551615 4506622 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=1779812 CPUtime=44.8 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 28923 0 0 0 4473 7 0 0 25 0 4 0 262487286 1822527488 260294 33554432000 4194304 4797429 140734255663168 18446744073709551615 4506576 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=1779812 CPUtime=45.05 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 32634 0 0 0 4496 9 0 0 25 0 4 0 262487286 1822527488 260294 33554432000 4194304 4797429 140734255663168 18446744073709551615 4506576 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 185.98
Current children cumulated vsize (KiB) 1779812

[startup+102.307 s]
/proc/loadavg: 7.87 6.26 6.13 9/184 12543
/proc/meminfo: memFree=28774000/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=1888100 CPUtime=390.52 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 334585 0 0 0 38986 66 0 0 25 0 4 0 262486683 1933414400 284130 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259680 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12537/statm: 472025 284130 509 148 0 466954 0
[pid=12537/tid=12539] ppid=12535 vsize=1888100 CPUtime=96.23 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 41738 0 0 0 9612 11 0 0 25 0 4 0 262487286 1933414400 284130 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259683 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=1888100 CPUtime=95.73 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 36078 0 0 0 9565 8 0 0 25 0 4 0 262487286 1933414400 284130 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236528 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=1888100 CPUtime=96.25 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 39801 0 0 0 9614 11 0 0 25 0 4 0 262487286 1933414400 284130 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259288 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 390.52
Current children cumulated vsize (KiB) 1888100

[startup+162.3 s]
/proc/loadavg: 8.03 6.59 6.25 9/184 12545
/proc/meminfo: memFree=28744028/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=1889256 CPUtime=630.18 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 337889 0 0 0 62951 67 0 0 25 0 4 0 262486683 1934598144 284666 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236528 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12537/statm: 472314 284666 509 148 0 467243 0
[pid=12537/tid=12539] ppid=12535 vsize=1889256 CPUtime=156.22 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 42338 0 0 0 15610 12 0 0 25 0 4 0 262487286 1934598144 284666 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236528 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=1889256 CPUtime=155.41 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 36650 0 0 0 15533 8 0 0 25 0 4 0 262487286 1934598144 284666 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259083 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=1889256 CPUtime=156.25 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 40190 0 0 0 15614 11 0 0 25 0 4 0 262487286 1934598144 284666 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259680 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 630.18
Current children cumulated vsize (KiB) 1889256

[startup+222.301 s]
/proc/loadavg: 8.15 6.88 6.37 9/184 12547
/proc/meminfo: memFree=28739900/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=1890412 CPUtime=869.85 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 341279 0 0 0 86918 67 0 0 25 0 4 0 262486683 1935781888 284766 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236524 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12537/statm: 472603 284766 509 148 0 467532 0
[pid=12537/tid=12539] ppid=12535 vsize=1890412 CPUtime=216.22 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 42879 0 0 0 21610 12 0 0 25 0 4 0 262487286 1935781888 284766 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259081 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=1890412 CPUtime=215.09 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 36903 0 0 0 21501 8 0 0 25 0 4 0 262487286 1935781888 284766 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259135 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=1890412 CPUtime=216.25 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 41202 0 0 0 21614 11 0 0 25 0 4 0 262487286 1935781888 284766 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259083 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 869.85
Current children cumulated vsize (KiB) 1890412

[startup+282.3 s]
/proc/loadavg: 8.18 7.11 6.48 9/184 12548
/proc/meminfo: memFree=28735008/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=1891568 CPUtime=1109.55 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 345298 0 0 0 110887 68 0 0 25 0 4 0 262486683 1936965632 285024 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259135 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/12537/statm: 472892 285024 509 148 0 467821 0
[pid=12537/tid=12539] ppid=12535 vsize=1891568 CPUtime=276.21 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 43417 0 0 0 27609 12 0 0 25 0 4 0 262487286 1936965632 285024 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236524 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=1891568 CPUtime=274.78 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 37123 0 0 0 27470 8 0 0 25 0 4 0 262487286 1936965632 285024 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259135 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=1891568 CPUtime=276.25 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 42245 0 0 0 27614 11 0 0 25 0 4 0 262487286 1936965632 285024 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259680 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1109.55
Current children cumulated vsize (KiB) 1891568

[startup+342.301 s]
/proc/loadavg: 8.19 7.30 6.58 9/184 12550
/proc/meminfo: memFree=28720448/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=1895036 CPUtime=1349.23 cores=1,3,5,7

################
# More data... #
################

[pid=12537] ppid=12535 vsize=2164260 CPUtime=2787.33 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 460174 0 0 0 278640 93 0 0 25 0 4 0 262486683 2216202240 360655 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236524 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12537/statm: 541065 360655 499 148 0 535994 0
[pid=12537/tid=12539] ppid=12535 vsize=2164260 CPUtime=696.18 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 68818 0 0 0 69600 18 0 0 25 0 4 0 262487286 2216202240 360655 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236524 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=2164260 CPUtime=693.97 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 61549 0 0 0 69384 13 0 0 25 0 4 0 262487286 2216202240 360655 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259265 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=2164260 CPUtime=696.26 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 65683 0 0 0 69610 16 0 0 25 0 4 0 262487286 2216202240 360655 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259106 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2787.33
Current children cumulated vsize (KiB) 2164260

[startup+762.306 s]
/proc/loadavg: 6.25 7.41 6.95 5/175 12578
/proc/meminfo: memFree=28710492/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=2286084 CPUtime=3027.04 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 493644 0 0 0 302606 98 0 0 25 0 4 0 262486683 2340950016 389474 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259288 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12537/statm: 571521 389474 447 148 0 566450 0
[pid=12537/tid=12539] ppid=12535 vsize=2286084 CPUtime=756.18 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 79484 0 0 0 75599 19 0 0 25 0 4 0 262487286 2340950016 389474 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236524 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=2286084 CPUtime=753.98 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 70028 0 0 0 75384 14 0 0 25 0 4 0 262487286 2340950016 389474 33554432000 4194304 4797429 140734255663168 18446744073709551615 4425966 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=2286084 CPUtime=756.27 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 77311 0 0 0 75609 18 0 0 25 0 4 0 262487286 2340950016 389474 33554432000 4194304 4797429 140734255663168 18446744073709551615 268660318210 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3027.04
Current children cumulated vsize (KiB) 2286084

[startup+822.301 s]
/proc/loadavg: 4.93 6.81 6.77 5/175 12579
/proc/meminfo: memFree=28699136/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=2230424 CPUtime=3266.75 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 505126 0 0 0 326575 100 0 0 25 0 4 0 262486683 2283954176 391424 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259265 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12537/statm: 557606 391424 447 148 0 552535 0
[pid=12537/tid=12539] ppid=12535 vsize=2230424 CPUtime=816.17 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 83892 0 0 0 81597 20 0 0 25 0 4 0 262487286 2283954176 391424 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259126 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=2230424 CPUtime=813.98 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 72563 0 0 0 81384 14 0 0 25 0 4 0 262487286 2283954176 391424 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236528 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=2230424 CPUtime=816.26 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 79967 0 0 0 81608 18 0 0 25 0 4 0 262487286 2283954176 391424 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236512 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3266.75
Current children cumulated vsize (KiB) 2230424

[startup+882.301 s]
/proc/loadavg: 4.34 6.30 6.59 5/175 12581
/proc/meminfo: memFree=28697964/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=2297116 CPUtime=3506.49 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 511565 0 0 0 350548 101 0 0 25 0 4 0 262486683 2352246784 391992 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259274 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12537/statm: 574279 391992 447 148 0 569208 0
[pid=12537/tid=12539] ppid=12535 vsize=2297116 CPUtime=876.18 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 85885 0 0 0 87597 21 0 0 25 0 4 0 262487286 2352246784 391992 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259680 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=2297116 CPUtime=873.98 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 74258 0 0 0 87384 14 0 0 25 0 4 0 262487286 2352246784 391992 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259265 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=2297116 CPUtime=876.27 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 81958 0 0 0 87608 19 0 0 25 0 4 0 262487286 2352246784 391992 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236524 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3506.49
Current children cumulated vsize (KiB) 2297116

[startup+942.301 s]
/proc/loadavg: 4.12 5.88 6.43 5/175 12583
/proc/meminfo: memFree=28677064/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=2316260 CPUtime=3746.21 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 522290 0 0 0 374519 102 0 0 25 0 4 0 262486683 2371850240 397609 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259288 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12537/statm: 579065 397609 447 148 0 573994 0
[pid=12537/tid=12539] ppid=12535 vsize=2316260 CPUtime=936.18 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 89103 0 0 0 93597 21 0 0 25 0 4 0 262487286 2371850240 397609 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259686 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=2316260 CPUtime=933.98 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 76611 0 0 0 93384 14 0 0 25 0 4 0 262487286 2371850240 397609 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259265 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=2316260 CPUtime=936.27 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 84677 0 0 0 93608 19 0 0 25 0 4 0 262487286 2371850240 397609 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259683 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3746.21
Current children cumulated vsize (KiB) 2316260

[startup+1002.3 s]
/proc/loadavg: 4.04 5.54 6.27 5/175 12584
/proc/meminfo: memFree=28523388/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=2557072 CPUtime=3985.92 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 589829 0 0 0 398477 115 0 0 25 0 4 0 262486683 2618441728 436249 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236524 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12537/statm: 639268 436249 448 148 0 634197 0
[pid=12537/tid=12539] ppid=12535 vsize=2557072 CPUtime=996.17 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 111810 0 0 0 99593 24 0 0 25 0 4 0 262487286 2618441728 436249 33554432000 4194304 4797429 140734255663168 18446744073709551615 4642256 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=2557072 CPUtime=993.98 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 82251 0 0 0 99382 16 0 0 25 0 4 0 262487286 2618441728 436249 33554432000 4194304 4797429 140734255663168 18446744073709551615 4531964 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=2557072 CPUtime=996.27 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 93238 0 0 0 99606 21 0 0 25 0 4 0 262487286 2618441728 436249 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236524 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3985.92
Current children cumulated vsize (KiB) 2557072

[startup+1062.3 s]
/proc/loadavg: 4.01 5.25 6.13 5/175 12586
/proc/meminfo: memFree=28437160/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=2577484 CPUtime=4225.64 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 632166 0 0 0 422441 123 0 0 25 0 4 0 262486683 2639343616 458061 33554432000 4194304 4797429 140734255663168 18446744073709551615 4423072 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12537/statm: 644371 458061 448 148 0 639300 0
[pid=12537/tid=12539] ppid=12535 vsize=2577484 CPUtime=1056.18 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 115771 0 0 0 105593 25 0 0 25 0 4 0 262487286 2639343616 458061 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259081 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=2577484 CPUtime=1053.98 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 88586 0 0 0 105380 18 0 0 25 0 4 0 262487286 2639343616 458061 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259135 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=2577484 CPUtime=1056.27 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 99543 0 0 0 105605 22 0 0 25 0 4 0 262487286 2639343616 458061 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259072 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4225.64
Current children cumulated vsize (KiB) 2577484

[startup+1122.3 s]
/proc/loadavg: 4.07 5.04 6.00 5/175 12588
/proc/meminfo: memFree=28390108/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=2647344 CPUtime=4465.37 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 663668 0 0 0 446409 128 0 0 25 0 4 0 262486683 2710880256 469244 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259099 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12537/statm: 661836 469244 448 148 0 656765 0
[pid=12537/tid=12539] ppid=12535 vsize=2647344 CPUtime=1116.18 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 120060 0 0 0 111592 26 0 0 25 0 4 0 262487286 2710880256 469244 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259081 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=2647344 CPUtime=1113.98 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 92576 0 0 0 111380 18 0 0 25 0 4 0 262487286 2710880256 469244 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259126 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=2647344 CPUtime=1116.28 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 111085 0 0 0 111604 24 0 0 25 0 4 0 262487286 2710880256 469244 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259099 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4465.37
Current children cumulated vsize (KiB) 2647344

[startup+1182.3 s]
/proc/loadavg: 4.02 4.85 5.87 5/175 12589
/proc/meminfo: memFree=28324940/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=2716972 CPUtime=4705.11 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 685937 0 0 0 470379 132 0 0 25 0 4 0 262486683 2782179328 485087 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236524 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12537/statm: 679243 485087 448 148 0 674172 0
[pid=12537/tid=12539] ppid=12535 vsize=2716972 CPUtime=1176.18 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 128593 0 0 0 117590 28 0 0 25 0 4 0 262487286 2782179328 485087 33554432000 4194304 4797429 140734255663168 18446744073709551615 4557691 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=2716972 CPUtime=1173.98 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 96430 0 0 0 117379 19 0 0 25 0 4 0 262487286 2782179328 485087 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236524 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=2716972 CPUtime=1176.27 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 117034 0 0 0 117603 24 0 0 25 0 4 0 262487286 2782179328 485087 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236524 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4705.11
Current children cumulated vsize (KiB) 2716972



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1206.1 s]
/proc/loadavg: 4.01 4.78 5.82 5/175 12590
/proc/meminfo: memFree=28311260/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=2789444 CPUtime=4800.21 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 693276 0 0 0 479888 133 0 0 25 0 4 0 262486683 2856390656 488843 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259288 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12537/statm: 697361 488843 448 148 0 692290 0
[pid=12537/tid=12539] ppid=12535 vsize=2789444 CPUtime=1199.99 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 130416 0 0 0 119970 29 0 0 25 0 4 0 262487286 2856390656 488843 33554432000 4194304 4797429 140734255663168 18446744073709551615 4424994 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=2789444 CPUtime=1197.79 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 97974 0 0 0 119760 19 0 0 25 0 4 0 262487286 2856390656 488843 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259072 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=2789444 CPUtime=1200.08 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 119073 0 0 0 119984 24 0 0 25 0 4 0 262487286 2856390656 488843 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236512 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 2789444

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1206.1 s]
/proc/loadavg: 4.01 4.78 5.82 5/175 12590
/proc/meminfo: memFree=28311260/32951124 swapFree=67111524/67111528
[pid=12537] ppid=12535 vsize=2789444 CPUtime=4800.21 cores=1,3,5,7
/proc/12537/stat : 12537 (techdemo) R 12535 12537 11524 0 -1 4202496 693276 0 0 0 479888 133 0 0 25 0 4 0 262486683 2856390656 488843 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259288 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/12537/statm: 697361 488843 448 148 0 692290 0
[pid=12537/tid=12539] ppid=12535 vsize=2789444 CPUtime=1199.99 cores=1,3,5,7
/proc/12537/task/12539/stat : 12539 (techdemo) R 12535 12537 11524 0 -1 4202560 130416 0 0 0 119970 29 0 0 25 0 4 0 262487286 2856390656 488843 33554432000 4194304 4797429 140734255663168 18446744073709551615 4424994 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=12537/tid=12540] ppid=12535 vsize=2789444 CPUtime=1197.79 cores=1,3,5,7
/proc/12537/task/12540/stat : 12540 (techdemo) R 12535 12537 11524 0 -1 4202560 97974 0 0 0 119760 19 0 0 25 0 4 0 262487286 2856390656 488843 33554432000 4194304 4797429 140734255663168 18446744073709551615 4259072 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=12537/tid=12541] ppid=12535 vsize=2789444 CPUtime=1200.08 cores=1,3,5,7
/proc/12537/task/12541/stat : 12541 (techdemo) R 12535 12537 11524 0 -1 4202560 119073 0 0 0 119984 24 0 0 25 0 4 0 262487286 2856390656 488843 33554432000 4194304 4797429 140734255663168 18446744073709551615 4236512 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 2789444

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1206.19
CPU time (s): 4800.32
CPU user time (s): 4798.9
CPU system time (s): 1.41478
CPU usage (%): 397.974
Max. virtual memory (cumulated for all children) (KiB): 2789444

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.9
system time used= 1.41478
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 693276
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 11031
involuntary context switches= 33670

runsolver used 1.80773 second user time and 4.28935 second system time

The end

Launcher Data 

Begin job on node107 at 2011-04-30 19:02:34
IDJOB=3293702
IDBENCH=88167
IDSOLVER=1584
FILE ID=node107/3293702-1304182953
RUNJOBID= node107-1304182824-12385
PBS_JOBID= 13172425
Free space on /tmp= 72512 MiB

SOLVER NAME= CryptoMiniSat TechDemo-Alpha-mt
BENCH NAME= SAT-Race-2010-CNF/hardware-verification/velev/11pipe_k.cnf
COMMAND LINE= ./techdemo --threads=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3293702-1304182953/watcher-3293702-1304182953 -o /tmp/evaluation-result-3293702-1304182953/solver-3293702-1304182953 -C 4800 -W 1300 -M 15500  ./techdemo --threads=4 HOME/instance-3293702-1304182953.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 932387240a0c17f4c88af52c4ca151c6
RANDOM SEED=2109238573

node107.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5359.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29950512 kB
Buffers:        400052 kB
Cached:        1879580 kB
SwapCached:          4 kB
Active:         574204 kB
Inactive:      2073344 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29950512 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:          110932 kB
Writeback:           0 kB
AnonPages:      367952 kB
Mapped:          15720 kB
Slab:           289868 kB
PageTables:       5428 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   982004 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72516 MiB
End job on node107 at 2011-04-30 19:22:42