CloudTest Summary Report

Here’s what you can expect to see

in a custom load test report from

Akamai Performance Engineering

CloudTest Summary Report 1

Contents

Executive Summary ................................................................................................................................................... 3

Introduction ................................................................................................................................................................. 3
Objectives/Goals ........................................................................................................................................................ 3
Test Scenarios .............................................................................................................................................................. 3
Results ......................................................................................................................................................................... 3
What We Learned ........................................................................................................................................................ 3
Recommendations ...................................................................................................................................................... 4
Conclusions .................................................................................................................................................................. 4
Test Results .................................................................................................................................................................. 5
Results from Date/Time ............................................................................................................................................. 5
Results Summary ........................................................................................................................................................ 5
Response Time Analysis ............................................................................................................................................ 6
Transaction Analysis .................................................................................................................................................... 7
Errors .......................................................................................................................................................................... 13
Send Rate / Hit Rate Analysis .................................................................................................................................. 14
Bandwidth Analysis .................................................................................................................................................. 15
Test Clip Analysis ...................................................................................................................................................... 16
Full Error Analysis ..................................................................................................................................................... 17
About Akamai .......................................................................................................................................................... 19

CloudTest Summary Report 2

Executive Summary

INTRODUCTION

This document presents the results of tests performed for “Customer” against the current production
environment. Domains or details that might help identify “Customer” have been anonymized. These tests
were conducted on the Akamai CloudTest platform.

OBJECTIVES/GOALS

The test was designed to measure system performance and scalability, while monitoring response time
metrics as user concurrency increases over time. This includes:

• Determine system scalability and performance while increasing user concurrency over time

• Determine system response times while increasing user concurrency over time

• Measure the impact of commercial load spikes on system performance while operating at peak load

TEST SCENARIOS

Nine test scenarios were created:

1) Scenario 1: 46% of Load 6) Scenario 6: 33.3% of Spike

2) Scenario 2: 24% of Load 7) Scenario 7: 3,000 Users
3) Scenario 3: 13% of Load 8) Scenario 8: 66.6% of Spike
4) Scenario 4: 17% of Load 9) Scenario 9: 33.3% of Spike
5) Scenario 5: 66.6% of Spike

Additional scenarios were added to this composition for a secondary commercial spike. This was intended
to initiate the first commercial spike, hit peak, and then initiate another commercial spike while the first was
ramping down. This new scenario sought to mimic behavior seen during the “event.”

Results

WHAT WE LEARNED
• Load is unable to scale effectively to peak base load without performance degradation

• The majority of issues occurred on the xxxxxx domain along with two xxxxxx domains

• The majority of the error types were HTTP 504, 502, 404, and 400

• “Transaction X” was the largest 90th percentile measure, by far, at 49.194 seconds

CloudTest Summary Report 3

 • Two xxxxxx calls to a development domain were second and third for longest 90th percentile time

• Transaction rates and response times degraded consistently as the test execution progressed

RECOMMENDATIONS

• Determine the cause of the degradation with “transaction X”

• Decide whether the development xxxxxx domain should be included in the test execution
and resources

• Reduce the system annotations to increase the performance of multiple dashboards; this is only
necessary when placing servers in an idle state

• Determine root cause for transaction rate reductions (specifically, xxxxxx) while executing
commercial spikes

CONCLUSIONS

Response time degraded continuously beyond xx,xxx users as concurrency increased, but leveled off
once it reached the initial ramp of xx,xxx users. Through the remainder of the test, average response time
increased each time a commercial spike was introduced into the system, but recovered as the commercial
users were ramped back down to average concurrency.

Significant increases in errors coincided with the commercial spikes. These primarily consisted of HTTP
504 (Gateway Timeout), HTTP 502 (Bad Gateway), HTTP 404 (Not Found), and HTTP 400 (Bad Request).
The majority of these errors occurred in the xxxxxx and xxxxxx domains. These errors can be attributed to
differences in the hardware on the seven in-play web servers. Three of the servers had older hardware,
with less load capacity (in terms of CPU and memory) than the other four. These were not recognized by
the load balancer, which continued to distribute the load equally, overloading those servers.

Two domains (XXXX and XXXX), each serving single-resource requests, were the slowest in terms of
response times. While these domains serve a very small percentage of the overall requests, it highlights
the possibility that third-party resource requests might have a detrimental impact on the performance of
certain pages.

The purpose of the test was to identify how the existing production site reacts while a large user
concurrency base is running on the system and specific types of users are added to the configuration
quickly. Various levels of commercial spikes were used to determine the configuration stability and
scalability. Transaction rates appear to degrade, over time, whether due to error rates, increased
concurrency, database struggles, or transaction X.

CloudTest Summary Report 4

Test Results

RESULTS FROM DATE/TIME

Test Start Date/Time: 04/17/2018, 05:16:59a.m.

Test End Date/Time: 04/17/2018, 07:35:15a.m.

Test Duration: 02:18:10.62

Description: Performance Testing the Production Infrastructure for “Customer”

RESULTS SUMMARY

High-level metrics/fundamentals from this test:

• HTTP requests sent: 193,800,623

• HTTP request errors: 350,504 (0.181%)
• Average response time: 222 ms
• Max response time: 278368 ms
• Total bytes sent: 520.7 GB
• Total bytes received: 2.1 TB
• Effective throughput (requests/second): 23,398 msgs/sec
• Effective throughput (bits/second): 2,543,365,201 bits/sec

The virtual users for this test were generated with the following geographic distribution:

Virtual Users by Location

CloudTest Summary Report 5

REPONSE TIME ANALYSIS

One of the primary metrics in a load test is the average response time. Average response times
provide a general idea of how the application is performing under load. The flatter the average
response time line (as the load/number of virtual users increases), the better.

The average response time chart shown below is an average of all the HTTP requests made during
the test. Response time average started to degrade at nearly xx,xxx users and continued until xx,xxx
concurrent users was attained. After that, response times only increased when subjected to commercial
spikes, but recovered once commercial was ramped back down.

Virtual Users vs. Average Collection Duration

When variances in response times are identified, it is important to identify the associated domains.
The chart below shows the average response times separated by domain over time. Both XXXX and
XXXX domains account for the larger response times seen below, but they correspond mainly with
commercial spikes.

Virtual Users vs. Average Collection Duration

CloudTest Summary Report 6

TRANSACTION ANALYSIS

Transactions represent a grouping of clip elements (HTTP requests, scripts, think times, etc.). Most tests
utilize transactions to closely represent the actual page load times that an end user would experience.
Ideally, the transaction completion time for each test clip should remain flat throughout the test.
Response times were very stable throughout the test execution and any notable increase appears to
coincide with the increased user concurrency spikes that were introduced at various intervals.

Virtual Users vs. Average Collection Duration

The chart below shows the 95th percentile measures throughout the duration of the test.

Virtual Users vs. Collection Duration 95th Percentile

The collection analysis widget on the next pages identifies each of the transactions used in the test
composition and statistics associated with each clip.

CloudTest Summary Report 7

 Collections Average Standard
Collection 90th Percentile
Completed Duration Deviation
Scenario1.01: Home Page 112,302 1.089 1.204 1.473
Scenario1.02: TransactionX 27,939 0.65 0.472 1.04
Scenario1.02: TransactionX 55,919 0.643 0.424 1.02
Scenario1.02: TransactionX 27,836 3.833 6.707 4.563
Scenario1.03: TransactionX 25,441 7.062 15.359 16.831
Scenario1.03: TransactionX 50,992 6.855 15.491 16.592
Scenario1.04: TransactionX 25,386 0.399 2.97 0.276
Scenario1.04: TransactionX 50,892 0.397 3.046 0.272
Scenario1.05: TransactionX 25,319 0.794 3.721 0.672
Scenario1.05: TransactionX 50,729 2.783 7.963 3.161
Scenario1.06: TransactionX 23,652 6.411 13.063 13.449
Scenario1.06: TransactionX 25,173 0.826 1.108 1.134
Scenario1.07: TransactionX 23,393 4.201 6.627 4.781
Scenario1.07: TransactionX 22,959 10.22 15.029 20.083
Scenario1.08: TransactionX 22,858 2.882 2.103 3.199
Scenario1.08: TransactionX 22,943 13.508 16.908 42.158
Scenario1.09: TransactionX 21,494 5.065 11.53 9.535
Scenario1.09: TransactionX 21,695 5.441 10.676 10.017
Scenario1.10: TransactionX 11,324 14.887 14.795 25.12
Scenario1.10: TransactionX 5,449 15.108 15.761 27.283
Scenario1.10: TransactionX 86,348 2.013 1.255 2.448
Scenario1.10: TransactionX 2,795 5.368 10.982 9.619
Scenario1.10: TransactionX 3,008 0.877 0.323 1.221
Scenario1.10: TransactionX 8,914 7.563 7.725 9.545
Scenario1.10: TransactionX 9,030 1.235 0.732 1.645
Scenario1.10: TransactionX 444,532 0.326 0.533 0.463
Scenario1.11-1: TransactionX 147,896 0.323 0.583 0.531
Scenario1.11-2: TransactionX 147,733 0.114 0.221 0.228
Scenario1.11-3: TransactionX 147,636 0.07 0.364 0.149
Scenario1.11-4: TransactionX 148,818 0.116 0.247 0.233
Scenario1.11-5: TransactionX 147,941 0.113 0.383 0.223
Scenario1.11-6: TransactionX 148,140 0.112 0.202 0.223
Scenario1.11: TransactionX 45,040 1.088 0.473 1.463
Scenario1.11: TransactionX 2,779 3.715 5.055 4.194
Scenario1.11: TransactionX 2,869 8.175 11.384 12.892
Scenario1.11: TransactionX 8,954 2.103 2.411 2.483
Scenario1.11: TransactionX 8,896 1.939 5.167 1.677
Scenario1.11: TransactionX 72,476 22.918 25.611 46.891
Scenario1.12: TransactionX 401,230 27.472 11.746 37.13
Scenario1.12: TransactionX 2,866 2.201 2.247 2.521
Scenario1.12: TransactionX 2,773 10.333 13.913 35.733
Scenario1.12: TransactionX 335 10.097 13.827 31.999
CloudTest Summary Report 8
 Collections Average Standard
Collection 90th Percentile
Completed Duration Deviation
Scenario1.12: TransactionX 443,669 0.317 1.298 0.44
Scenario1.12a: TransactionX 1,694,372 7.236 7.883 11.842
Scenario1.13: TransactionX 2,681 3.777 10.081 5.289
Scenario1.13: TransactionX 2,792 3.913 8.219 5.304
Scenario1.13: TransactionX 443,130 0.337 0.456 0.48
Scenario1.13: TransactionX 335 3.064 7.625 2.349
Scenario1.13: TransactionX 400,459 0.344 0.621 0.483
Scenario1.14-1: TransactionX 133,558 0.427 0.574 0.677
Scenario1.14-2: TransactionX 132,967 0.18 0.268 0.332
Scenario1.14-3: TransactionX 133,432 0.192 0.278 0.344
Scenario1.14-4: TransactionX 133,514 0.112 0.166 0.228
Scenario1.14-5: TransactionX 133,114 0.11 0.236 0.22
Scenario1.14-6: TransactionX 133,422 0.109 0.153 0.223
Scenario1.14: TransactionX 335 3.331 9.01 10.399
Scenario1.14: TransactionX 442,833 0.508 1.599 0.653
Scenario1.15: TransactionX 335 2.898 7.023 3.549
Scenario1.15: TransactionX 442,605 0.035 0.059 0.065
Scenario1.15: TransactionX 399,625 0.334 1.442 0.459
Scenario1.16: TransactionX 399,111 0.324 0.523 0.457
Scenario1.16: TransactionX 335 9.088 11.881 30.8
Scenario1.17: TransactionX 178 5.51 10.006 20.799
Scenario1.17: TransactionX 157 1.454 4.769 0.765
Scenario1.17: TransactionX 398,870 0.441 1.62 0.519
Scenario1.18: TransactionX 157 3.763 9.102 3.145
Scenario1.19: TransactionX 335 5.182 11.459 19.999
Scenario1.99: TransactionX 34,210 0.722 1.431 1.043
Scenario2.01: TransactionX 69,351 1.086 1.197 1.46
Scenario2.02: TransactionX 69,187 0.632 0.466 0.984
Scenario2.03: TransactionX 62,666 7.242 14.977 16.743
Scenario2.04: TransactionX 62,551 0.521 3.717 0.339
Scenario2.05: TransactionX 62,384 3.692 10.057 5.148
Scenario2.06: TransactionX 26,230 7.993 14.007 17.617
Scenario2.06: TransactionX 28,773 0.826 1.354 1.117
Scenario2.07: TransactionX 25,786 4.767 7.969 5.322
Scenario2.07: TransactionX 25,739 11.94 16.478 21.868
Scenario2.08: TransactionX 25,618 3.079 2.857 3.423
Scenario2.08: TransactionX 25,297 19.371 18.898 49.031
Scenario2.09: TransactionX 22,994 7.164 13.971 17.139
Scenario2.09: TransactionX 23,433 8.098 14.168 17.892
Scenario2.10: TransactionX 44,657 2.282 2.095 2.611
Scenario2.11: TransactionX 34,195 31.589 30.5 61.474
Scenario2.12: TransactionX 64,569 9.525 11.566 16.482
CloudTest Summary Report 9
 Collections Average Standard
Collection 90th Percentile
Completed Duration Deviation
Scenario2.13: TransactionX 64,290 0.301 0.836 0.426
Scenario2.14: TransactionX 64,134 0.366 1.912 0.465
Scenario2.15: TransactionX 63,919 0.261 0.745 0.361
Scenario2.16: TransactionX 63,745 0.586 2.001 0.783
Scenario2.17: TransactionX 63,600 0.038 0.438 0.061
Scenario2.99: TransactionX 29,943 0.947 3.07 1.231
Scenario3.01: TransactionX 27,281 1.117 1.26 1.523
Scenario3.02: TransactionX 6,646 3.35 5.123 4.026
Scenario3.02: TransactionX 20,248 0.669 0.432 1.081
Scenario3.03: TransactionX 18,635 5.077 12.763 10.126
Scenario3.04: TransactionX 18,336 0.307 2.453 0.243
Scenario3.05: TransactionX 18,050 1.537 4.715 1.633
Scenario3.06: TransactionX 4,497 5.511 11.178 9.934
Scenario3.06: TransactionX 5,024 0.953 0.518 1.433
Scenario3.07: TransactionX 4,164 6.528 10.899 10.789
Scenario3.07: TransactionX 4,597 8.016 11.497 12.825
Scenario3.08: TransactionX 4,523 2.844 1.914 3.105
Scenario3.08: TransactionX 4,020 9.698 14.711 38.099
Scenario3.09: TransactionX 5,160 16.447 15.996 29.104
Scenario3.09: TransactionX 3,841 3.954 8.285 6.532
Scenario3.09: TransactionX 4,264 5.391 10.117 9.907
Scenario3.10: TransactionX 2,061 14.473 12.279 24.799
Scenario3.10: TransactionX 18,250 1.112 0.484 1.484
Scenario3.10: TransactionX 1,072 5.343 8.529 9.999
Scenario3.10: TransactionX 1,123 0.97 0.581 1.365
Scenario3.10: TransactionX 23,713 10.453 11.907 17.845
Scenario3.10: TransactionX 3,974 8.068 8.66 10.292
Scenario3.10: TransactionX 3,921 1.53 0.565 1.992
Scenario3.11: TransactionX 1,017 6.436 11.003 10.799
Scenario3.11: TransactionX 1,062 8.635 12.226 14.333
Scenario3.11: TransactionX 3,899 2.091 2.144 2.433
Scenario3.11: TransactionX 3,970 2.132 5.515 1.915
Scenario3.11: TransactionX 387,129 0.198 0.33 0.388
Scenario3.12: TransactionX 202,349 1.346 1.924 2.312
Scenario3.12: TransactionX 183,981 2.003 3.548 3.075
Scenario3.12: TransactionX 1,060 2.273 3.144 2.514
Scenario3.12: TransactionX 1,012 9.579 13.365 29.2
Scenario3.12: TransactionX 374 8.481 11.248 26.933
Scenario3.13: TransactionX 974 3.931 7.093 7.449
Scenario3.13: TransactionX 1,015 5.849 9.876 10.307
Scenario3.13: TransactionX 374 3.269 6.877 6.099
Scenario3.13: TransactionX 381,487 0.321 0.46 0.428
CloudTest Summary Report 10
 Collections Average Standard
Collection 90th Percentile
Completed Duration Deviation
Scenario3.14: TransactionX 373 1.512 4.779 0.895
Scenario3.14: TransactionX 380,960 0.698 1.951 1.143
Scenario3.15: TransactionX 373 2.635 6.953 1.779
Scenario3.16: TransactionX 371 6.937 9.313 17.599
Scenario3.17: TransactionX 179 5.148 10.358 11.599
Scenario3.17: TransactionX 191 2.197 6.334 0.899
Scenario3.18: TransactionX 191 2.866 6.521 2.499
Scenario3.19: TransactionX 368 3.96 8.949 11.599
Scenario3.99: TransactionX 9,820 0.818 1.926 1.185
Scenario4.01: TransactionX 239,695 6.616 13.598 16.578
Scenario4.02: TransactionX 214,381 0.101 0.437 0.172
Scenario4.02: TransactionX 23,453 1.124 4.857 0.922
Scenario4.03: TransactionX 212,547 0.602 3.303 0.529
Scenario4.04: TransactionX 204,839 3.851 8.479 4.445
Scenario4.05: TransactionX 10,801 3.732 4.854 5.663
Scenario4.05: TransactionX 139,270 1.616 4.238 1.516
Scenario4.05: TransactionX 40,494 4.116 6.81 3.934
Scenario4.06: TransactionX 40,102 3.497 3.971 3.721
Scenario4.06: TransactionX 10,760 0.714 1.841 1.786
Scenario4.06: TransactionX 136,623 19.437 19.737 53.56
Scenario4.07: TransactionX 10,136 6.239 9.984 10.694
Scenario4.07: TransactionX 175,185 1.444 3.325 1.44
Scenario4.08: TransactionX 1,013,309 0.345 0.69 0.486
Scenario4.08: TransactionX 770,799 0.454 0.439 0.775
Scenario4.09: TransactionX 1,008,249 0.363 0.37 0.626
Scenario4.09: TransactionX 768,249 0.301 0.36 0.517
Scenario4.10: TransactionX 730,964 3.631 10.381 8.038
Scenario4.11: TransactionX 728,780 0.335 0.416 0.575
Scenario4.99: TransactionX 105,933 6.877 14.941 16.619
Scenario5.01: TransactionX 4,642 0.883 0.844 1.239
Scenario5.02: TransactionX 4,341 2.563 2.609 3.812
Scenario5.03: TransactionX 15,739 2.604 1.524 3.944
Scenario5.04: TransactionX 14,568 2.917 2.273 3.757
Scenario6.01: TransactionX 50,527 1.109 1.374 1.468
Scenario6.02: TransactionX 11,180 11.754 23.157 23.999
Scenario6.02: TransactionX 37,539 0.646 0.364 1.028
Scenario6.03: TransactionX 34,605 5.304 13.158 10.892
Scenario6.04: TransactionX 34,304 0.309 2.504 0.236
Scenario6.05: TransactionX 33,915 1.566 4.879 1.651
Scenario6.06: TransactionX 8,860 5.49 10.651 9.983
Scenario6.06: TransactionX 9,410 0.936 0.583 1.357
Scenario6.07: TransactionX 8,590 4.697 8.188 5.626

CloudTest Summary Report 11

 Collections Average Standard
Collection 90th Percentile
Completed Duration Deviation
Scenario6.07: TransactionX 8,647 8.109 12.133 12.759
Scenario6.08: TransactionX 8,383 15.862 14.947 28.124
Scenario6.08: TransactionX 8,565 2.869 2.059 3.102
Scenario6.08: TransactionX 8,399 11.678 16.183 41.106
Scenario6.09: TransactionX 7,961 4.382 8.679 8.958
Scenario6.09: TransactionX 8,055 5.702 10.922 10.289
Scenario6.09: TransactionX 30,820 1.086 0.455 1.47
Scenario6.10: TransactionX 3,347 15.525 15.209 27.371
Scenario6.10: TransactionX 1,728 5.395 10.677 9.399
Scenario6.10: TransactionX 1,822 0.913 0.347 1.289
Scenario6.10: TransactionX 6,362 7.695 7.985 9.727
Scenario6.10: TransactionX 6,371 1.406 0.798 1.892
Scenario6.10: TransactionX 19,120 0.287 0.469 0.482
Scenario6.10: TransactionX 19,548 0.711 1.072 1.059
Scenario6.11: TransactionX 331,656 1.85 2.053 3.224
Scenario6.11: TransactionX 301,013 2.063 2.579 3.451
Scenario6.11: TransactionX 1,706 4.412 6.205 5.373
Scenario6.11: TransactionX 1,744 8.52 12.34 13.155
Scenario6.11: TransactionX 6,324 1.96 2.014 2.298
Scenario6.11: TransactionX 6,346 2.17 5.744 1.816
Scenario6.12: TransactionX 300,526 1.517 2.189 2.566
Scenario6.12: TransactionX 1,740 2.183 2.06 2.539
Scenario6.12: TransactionX 331,266 0.192 0.235 0.397
Scenario6.12: TransactionX 1,699 10.565 14.114 36.799
Scenario6.12: TransactionX 284 10.086 13.142 30.666
Scenario6.13: TransactionX 1,630 4.643 10.648 9.054
Scenario6.13: TransactionX 300,068 0.528 0.528 0.973
Scenario6.13: TransactionX 1,678 5.398 9.455 10.19
Scenario6.13: TransactionX 284 2.322 6.055 2.219
Scenario6.13: TransactionX 324,971 0.955 0.688 1.608
Scenario6.14: TransactionX 284 2.912 7.391 8.949
Scenario6.14: TransactionX 299,643 0.11 0.161 0.224
Scenario6.15: TransactionX 284 2.431 6.383 1.719
Scenario6.15: TransactionX 291,627 0.822 0.776 1.221
Scenario6.16: TransactionX 284 8.219 10.629 25
Scenario6.17: TransactionX 138 5.635 10.029 14.799
Scenario6.17: TransactionX 145 2.042 5.683 1.449
Scenario6.18: TransactionX 145 2.942 6.741 2.559
Scenario6.19: TransactionX 283 4.325 8.496 14.533
Scenario6.99: TransactionX 17,311 0.76 1.418 1.103

CloudTest Summary Report 12

ERRORS
Errors are an easy way to identify where the problems are in a test. Network issues, infrastructure,
hardware, application, and any number of other sources can cause errors. Full error details are
available in the Full Error Analysis section in the last pages of this result. Errors consisted mainly of
connection timeouts and HTTP 504, 502, 404, and 400 errors. (Usually, there are legends for the
following charts, but they have been removed from this report in the interest of anonymity).

Error Count

Error Count vs. Errors by Type over Time

Errors by Type over Time

CloudTest Summary Report 13

SEND RATE / HIT RATE ANALYSIS

Another important metric in any load test is the throughput (i.e., send rate or hit rate) of the test.
This measures how many raw HTTP requests can be processed in a given interval (e.g., per second
or per minute) by the target application. In a perfectly scalable application, the send rate increases
in a straight line with the virtual user ramp. Send rate increased linearly as virtual user concurrency
increased, indicating that throughput is not a concern at this point in time. (Usually, there are legends
for the following charts, but they have been removed from this report in the interest of anonymity).

Virtual Users vs. Send Rate

The XXXX domain accounted for the majority of the send rate data below.

Errors by Type over Time

CloudTest Summary Report 14

BANDWIDTH ANALYSIS

Bandwidth is a potential bottleneck that can become maxed out very quickly. This is especially
true if an application has large downloads, lots of page resources, or does not properly utilize a
CDN. Bandwidth does not appear to be an issue at this point in time. This metric should always be
revisited after each test in order to verify there are no changes that may have caused a bottleneck or
degradation. (Usually, there are legends for the following charts, but they have been removed from this
report in the interest of anonymity).

Bandwidth Usage

Bandwish Usage by Domain

CloudTest Summary Report 15

TEST CLIP ANALYSIS

The chart below shows the completion time for each of the user scenarios (test cases) over the course
of the test. Ideally, the completion time for each test clip should remain flat throughout the test.
(Usually, there are legends for the following charts, but they have been removed from this report in the
interest of anonymity).

Average Clip Duration

The chart below shows how many clips were completed through the course of the test. Note that the
test execution itself incorporated spikes in traffic, which you can see accurately reflected in the chart at
the corresponding times.

Clip Completed

The clip analysis widget on the next page identifies each of the clips used in the test composition and
statistics associated with each clip.

CloudTest Summary Report 16

 Clip Analysis
Component Clips Average Maximum
Bytes Sent Bytes Received Errors
Hierarchy Completed Duration Duration
Composition 493,455 658.035 s 8,187.244 s 401,348,098,380 1,732,113,725,029 113,723
Scenario 1 94,048 538.724 s 8,001.404 s 131,190,717,498 311,124,204,116 17,857
Scenario 2 69,531 183.291 s 670.233 s 42,456,494,993 173,923,240,388 29,190
Scenario 3 159,430 309.675 s 639.730 s 30,531,767,971 319,869,928,469 35,400
Scenario 4 24,243 1,264.178 s 8,168.812 s 39,937,195,294 419,228,478,054 3,670
Scenario 5 95,605 1,264.259 s 8,187.244 s 93,538,651,159 414,184,575,091 19,716
Scenario 6 4,812 3,919.446 s 8,108.647 s 444,643,336 5,222,159,216 33
Scenario 7 45,786 907.517 s 8,117.809 s 63,248,628,129 88,561,139,695 7,857

FULL ERROR ANALYSIS

The table below shows a detailed breakdown of error type and messages that resulted in error.

Error Requests Error Count

193,968,161 337,620
DomainX 973,271 64,291
Connection reset (java.net.SocketException) 271
Connection timeout of 60,000 ms exceeded. 62,147
Send was completed, but no response was received within the Socket Read
Timeout limit of 120,000 ms.
DomainX 1,049,386 23
Connection timeout of 60,000 ms exceeded. 15
HTTP 400 - Bad Request 5
DomainX 295,788 37
Connection timeout of 60,000 ms exceeded. 37
DomainX 60,762,484 31,124
Connection reset (java.net.SocketException) 1
Connection timeout of 60,000 ms exceeded. 89
Failed to Process Transaction X 14
HTTP 400 - Bad Request 1
HTTP 403 - Forbidden 3,135
HTTP 500 - Internal Server Error 986
HTTP 502 - Bad Gateway 331
HTTP 503 - Service Unavailable 17
HTTP 504 - Gateway Timeout 10,672

CloudTest Summary Report 17

 Error Requests Error Count

Send was completed, but no response was received within the Socket Read
67
Timeout limit of 120,000 ms.
Send was completed, but the connection was broken before the response
4
was received.
The SSL server certificate for domainx cannot be verified. (com.soasta.
8
common.exceptions.CommonException)
Unable to set value of Property - check events for specific properties 10,130
Unable to set value of the Property account” - JSON response was invalid.” 5,185
DomainX 1,980 32
HTTP 400 - Bad Request 2
HTTP 500 - Internal Server Error 6
HTTP 503 - Service Unavailable 24
DomainX 743,308 8
Connection timeout of 60000 ms exceeded. 8
DomainX 38,695 1,649
Connection reset (java.net.SocketException) 4
Connection timeout of 60,000 ms exceeded. 1,429
DomainX 5,317,556 93,871
Connection reset (java.net.SocketException) 377
Connection timeout of 60,000 ms exceeded. 92,016
Send was completed, but no response was received within the Socket Read
1
Timeout limit of 120,000 ms.
Send was completed, but the connection was broken before the response
7
was received.
DomainX 124,785,689 146,581
Custom Validation Error Message 1 24
Custom Validation Error Message 2 3,752
Custom Validation Error Message 3 14,016
Custom Validation Error Message 4 3
Connection timeout of 60,000 ms exceeded. 1,102
Custom Validation Error Message 5 6
HTTP 400 - Bad Request 11,596
HTTP 401 - Unauthorized 5
HTTP 417 - Expectation Failed 1
HTTP 500 - Internal Server Error 3,420
HTTP 502 - Bad Gateway 10,319
HTTP 503 - Service Unavailable 5,494
HTTP 504 - Gateway Timeout 49,335

CloudTest Summary Report 18

 Error Requests Error Count

HTTP Error: 404 Not Found 11,602

Custom Validation Error Message 6 2,043
Custom Validation Error Message 7 3
Custom Validation Error Message 8 611
Send was completed, but no response was received within the Socket Read
115
Timeout limit of 120,000 ms.
Send was completed, but the connection was broken before the response
2
was received.
The SSL server certificate for domainX cannot be verified. (com.soasta.
9
common.exceptions.CommonException)
Unable to set value of Property - check events for specific properties 7,597
Unable to set value of the Property account" - JSON response was invalid." 20
Unable to set value of the Property X" - JSON response was invalid." 1
Custom Validation Error Message 9 5,345
Custom Validation Error Message 10 2,052
Custom Validation Error Message 11 3,264

Start Testing Today

Ready to load test your website and apps? Visit Akamai.com/cloudtest for more information.

Learn More

Akamai secures and delivers digital experiences for the world’s largest companies. Akamai’s intelligent edge platform
surrounds everything, from the enterprise to the cloud, so customers and their businesses can be fast, smart, and secure.
Top brands globally rely on Akamai to help them realize competitive advantage through agile solutions that extend the power
of their multi-cloud architectures. Akamai keeps decisions, apps, and experiences closer to users than anyone — and attacks
and threats far away. Akamai’s portfolio of edge security, web and mobile performance, enterprise access, and video delivery
solutions is supported by unmatched customer service, analytics, and 24/7/365 monitoring. To learn why the world’s top brands
trust Akamai, visit www.akamai.com, blogs.akamai.com, or @Akamai on Twitter. You can find our global contact information
at www.akamai.com/locations. Published 05/19.

CloudTest Summary Report 19