# Autoscaling Microservices
**System Architecture**

![img.png](Open-Prediction Scaler/keda/img.png)


**Some Load Test Results**

|                                              | Statistics                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|----------------------------------------------| |
| **Open-Prediction Scaler (Proposed Scaler)** | <section class="col-md-12 table-responsive"><table id="statisticsTable" class="table table-bordered table-condensed tablesorter tablesorter-blue" role="grid"><colgroup class="tablesorter-colgroup"><col style="width: 6%;"><col style="width: 7.2%;"><col style="width: 4.8%;"><col style="width: 5.1%;"><col style="width: 6.5%;"><col style="width: 4.4%;"><col style="width: 4.9%;"><col style="width: 6%;"><col style="width: 6.5%;"><col style="width: 6.5%;"><col style="width: 6.5%;"><col style="width: 9.5%;"><col style="width: 7%;"><col style="width: 4.8%;"></colgroup><thead><tr class="tablesorter-no-sort tablesorter-headerRow" role="row"><th data-sorter="false" colspan="1" data-column="0" class="tablesorter-header sorter-false tablesorter-headerUnSorted" scope="col" role="columnheader" aria-disabled="true" unselectable="on" aria-sort="none" style="user-select: none;"><div class="tablesorter-header-inner">Requests</div></th><th data-sorter="false" colspan="3" data-column="1" class="tablesorter-header sorter-false tablesorter-headerUnSorted" scope="col" role="columnheader" aria-disabled="true" unselectable="on" aria-sort="none" style="user-select: none;"><div class="tablesorter-header-inner">Executions</div></th><th data-sorter="false" colspan="7" data-column="4" class="tablesorter-header sorter-false tablesorter-headerUnSorted" scope="col" role="columnheader" aria-disabled="true" unselectable="on" aria-sort="none" style="user-select: none;"><div class="tablesorter-header-inner">Response Times (ms)</div></th><th data-sorter="false" colspan="1" data-column="11" class="tablesorter-header sorter-false tablesorter-headerUnSorted" scope="col" role="columnheader" aria-disabled="true" unselectable="on" aria-sort="none" style="user-select: none;"><div class="tablesorter-header-inner">Throughput</div></th><th data-sorter="false" colspan="2" data-column="12" class="tablesorter-header sorter-false tablesorter-headerUnSorted" scope="col" role="columnheader" aria-disabled="true" unselectable="on" aria-sort="none" style="user-select: none;"><div class="tablesorter-header-inner">Network (KB/sec)</div></th></tr><tr role="row" class="tablesorter-headerRow"><th data-column="0" class="tablesorter-header tablesorter-headerAsc" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="ascending" aria-label="Label: Ascending sort applied, activate to apply a descending sort" style="user-select: none;"><div class="tablesorter-header-inner">Label</div></th><th data-column="1" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="#Samples: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">#Samples</div></th><th data-column="2" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="FAIL: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">FAIL</div></th><th data-column="3" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Error %: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Error %</div></th><th data-column="4" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Average: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Average</div></th><th data-column="5" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Min: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Min</div></th><th data-column="6" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Max: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Max</div></th><th data-column="7" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Median: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Median</div></th><th data-column="8" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="90th pct: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">90th pct</div></th><th data-column="9" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="95th pct: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">95th pct</div></th><th data-column="10" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="99th pct: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">99th pct</div></th><th data-column="11" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Transactions/s: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Transactions/s</div></th><th data-column="12" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Received: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Received</div></th><th data-column="13" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Sent: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Sent</div></th></tr></thead><tbody class="tablesorter-no-sort"><tr role="row"><td>Calculate fibonacci 33</td><td>29505</td><td>13</td><td>0.04%</td><td>7801.81</td><td>5</td><td>39996</td><td>4788.00</td><td>16162.80</td><td>17049.95</td><td>20179.96</td><td>78.58</td><td>11.06</td><td>11.89</td></tr></tbody><tbody class="tablesorter-no-sort"><tr role="row"><td>Calculate fibonacci 35</td><td>6440</td><td>598</td><td>9.29%</td><td>71032.96</td><td>5</td><td>180123</td><td>51518.50</td><td>168930.60</td><td>180010.00</td><td>180022.00</td><td>13.66</td><td>2.11</td><td>2.07</td></tr></tbody></table></section>   | 
| **Prometheus Scaler (KEDA)**                 | <section class="col-md-12 table-responsive"><table id="statisticsTable" class="table table-bordered table-condensed tablesorter tablesorter-blue" role="grid"><colgroup class="tablesorter-colgroup"><col style="width: 6%;"><col style="width: 7.2%;"><col style="width: 4.8%;"><col style="width: 5.1%;"><col style="width: 6.5%;"><col style="width: 4.4%;"><col style="width: 4.9%;"><col style="width: 6%;"><col style="width: 6.5%;"><col style="width: 6.5%;"><col style="width: 6.5%;"><col style="width: 9.5%;"><col style="width: 7%;"><col style="width: 4.8%;"></colgroup><thead><tr class="tablesorter-no-sort tablesorter-headerRow" role="row"><th data-sorter="false" colspan="1" data-column="0" class="tablesorter-header sorter-false tablesorter-headerUnSorted" scope="col" role="columnheader" aria-disabled="true" unselectable="on" aria-sort="none" style="user-select: none;"><div class="tablesorter-header-inner">Requests</div></th><th data-sorter="false" colspan="3" data-column="1" class="tablesorter-header sorter-false tablesorter-headerUnSorted" scope="col" role="columnheader" aria-disabled="true" unselectable="on" aria-sort="none" style="user-select: none;"><div class="tablesorter-header-inner">Executions</div></th><th data-sorter="false" colspan="7" data-column="4" class="tablesorter-header sorter-false tablesorter-headerUnSorted" scope="col" role="columnheader" aria-disabled="true" unselectable="on" aria-sort="none" style="user-select: none;"><div class="tablesorter-header-inner">Response Times (ms)</div></th><th data-sorter="false" colspan="1" data-column="11" class="tablesorter-header sorter-false tablesorter-headerUnSorted" scope="col" role="columnheader" aria-disabled="true" unselectable="on" aria-sort="none" style="user-select: none;"><div class="tablesorter-header-inner">Throughput</div></th><th data-sorter="false" colspan="2" data-column="12" class="tablesorter-header sorter-false tablesorter-headerUnSorted" scope="col" role="columnheader" aria-disabled="true" unselectable="on" aria-sort="none" style="user-select: none;"><div class="tablesorter-header-inner">Network (KB/sec)</div></th></tr><tr role="row" class="tablesorter-headerRow"><th data-column="0" class="tablesorter-header tablesorter-headerAsc" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="ascending" aria-label="Label: Ascending sort applied, activate to apply a descending sort" style="user-select: none;"><div class="tablesorter-header-inner">Label</div></th><th data-column="1" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="#Samples: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">#Samples</div></th><th data-column="2" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="FAIL: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">FAIL</div></th><th data-column="3" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Error %: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Error %</div></th><th data-column="4" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Average: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Average</div></th><th data-column="5" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Min: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Min</div></th><th data-column="6" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Max: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Max</div></th><th data-column="7" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Median: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Median</div></th><th data-column="8" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="90th pct: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">90th pct</div></th><th data-column="9" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="95th pct: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">95th pct</div></th><th data-column="10" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="99th pct: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">99th pct</div></th><th data-column="11" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Transactions/s: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Transactions/s</div></th><th data-column="12" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Received: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Received</div></th><th data-column="13" class="tablesorter-header tablesorter-headerUnSorted" tabindex="0" scope="col" role="columnheader" aria-disabled="false" aria-controls="statisticsTable" unselectable="on" aria-sort="none" aria-label="Sent: No sort applied, activate to apply an ascending sort" style="user-select: none;"><div class="tablesorter-header-inner">Sent</div></th></tr></thead><tbody class="tablesorter-no-sort"><tr role="row"><td>Calculate fibonacci 33</td><td>25470</td><td>1</td><td>0.00%</td><td>9190.15</td><td>7</td><td>74685</td><td>7534.00</td><td>15678.70</td><td>23824.95</td><td>45810.81</td><td>68.61</td><td>9.65</td><td>10.45</td></tr></tbody><tbody class="tablesorter-no-sort"><tr role="row"><td>Calculate fibonacci 35</td><td>5595</td><td>1900</td><td>33.96%</td><td>90738.57</td><td>303</td><td>229345</td><td>81812.00</td><td>180009.00</td><td>180016.00</td><td>181012.00</td><td>10.20</td><td>1.96</td><td>1.55</td></tr></tbody></table></section> |


