You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This subsection presents a graphical illustration of some chosen performance metrics of SmartNotes application running on Google App Engine. Taking into account the concurrency factor of incoming requests and the rate in which the connection rate becomes increased the factors that will be analysed include:
34
+
Performance testing form does strongly depend on the research scope. Testing the application performance is only a general name for the process that should become narrowed to a single parameter or set of well defined properties. The usage of wright tool does also matter just as making appropriate assumptions and choosing wright model used as otherwise measured values could not match the real application characteristics.
35
+
36
+
The analyses done in this chapter are fallowing the guidelines as defined in~\cite{gae_best_practises_plus_load_tests} for testing applications that use Google App Engine:
35
37
\begin{itemize}
36
-
\item{Minimum reply rate - }
37
-
\item{Average reply rate - }
38
-
\item{Maximum reply rate -}
39
-
\item{X}
38
+
\item{Use production system. Most of web frameworks provide a development including sever that can be easily run on the local machine. What speedups the development at one side cant be used for production purposes. This environment divers significantly from the deployment system and cant be to give much information of final system performance.}
39
+
\item{Gradual ramp up. The goal of load tests is to probe the system reaction for a certain input parameter sets. Therefore its strongly desirable to make it remind the realistic cases. Besides the resources of Google App Engine are granted only when the application needs them. That motivates a wrap up just before the wright test takes place.}
40
+
\item{Realistic load. Is seams pointless to run test for a situation that the system is unlikely to reach.}
40
41
\end{itemize}
41
-
The tests were performed for main SmartNotes web page by using tools like \texttt{httperf} and \texttt{autobench} and collected data allowed to make graphs shown in Figure
42
+
The fallowing part of this subsection presents a graphical illustration for selected performance metrics of SmartNotes application running on Google App Engine. Taking into account the parameters such as the rate of opened connections and reached request rate the analyses in the first step will focus on the variations of the response rate and will be finalised with a comparison of the average response time between different techniques. This includes a internalized dynamic content, cached page and same page served as static content. The first is a most classical case when some parts of page depending on state may differ, second one makes use of highly popular caching technique and lastly the special infrastructure devoted only for serving static content. All of these will become wider dissed by analysing corresponding results.
43
+
44
+
The view on dashboard of SmartNotes application shown in Figure~\ref{fig:sn_dash_view} exposes a plot of request rate timeline for two different scenarios. The first one marked as~\textit{(a)} illustrates a regular usage case with none anomalies wheres the second figure~\textit{(b)} presents a case when outgoing bandwidth became exceeded. This is most basic way of tracking the application performance and status. Providing couple of other tools which were discussed in Section~\ref{sec:gae_general} admin interface is truly functional yet easy to use. Providing a logs browser it leaves a place where user can place his application specific informations on several logging levels.
\subfigure[\textbf{Regular situation with average of about one\newline request per second}.]{\label{fig:dash_normal}\includegraphics[scale=0.18]{img/DASH_Regular_stats.png}}
48
+
\subfigure[\textbf{Application reaching the outgoing bandwidth limit on heavy traffic situation with the maximum of 170 requests per second}.]{\label{fig:dash_out_of_bandwith}\includegraphics[scale=0.2]{img/DASH_Out_of_bandwith.png}}
47
49
\end{center}
48
-
\caption{SmartNotes performence statistics.}
49
-
\label{fig:sn_web_interface}
50
+
\caption{The view on the public web-based SmartNotes interface with basic information regarding the project and authentication.}
51
+
\label{fig:sn_dash_view}
52
+
\end{figure}
53
+
The system of quotas of introduced by Google provides to phase limitations the daily limits and narrowed most common to minute quotas.The second one is a protection for load high peaks or any malicious or testing software that without it could let to letting the target application out of resource in just a couple of minutes. Thats exactly the case uncounted during the tests by receiving responses with \texttt{503 Service Unavailable} status codes when crossing the short tame quota and \texttt{403 Forbidden} responses after running over the daily limits.
54
+
55
+
The author has chosen Open Source tools like \texttt{httperf} from HP and \texttt{autobench} wrapper to carry out the tests for the main SmartNotes web page. This tools were chosen beyond others researched\footnote{That list includes tools like \texttt{siege} and \texttt{ab} which are popular Open source load testing programs.} due support for a gradual increase of connection rate and extended output. Each of the tests which results are presented in Figures~\ref{fig:sm_benchmark_normal}, \ref{fig:sm_benchmark_cache}, \ref{fig:sm_benchmark_static} and \ref{fig:sm_resp_time_comapre} uses the same set of input parameters:
56
+
\begin{itemize}
57
+
\item{500 connections per test. This connected with twenty second connection timeout allowed to reach up to 400 concurrent connections. }
58
+
\item{3 requests per connection. This implies a total number of 1500 requests per test. This parameter was chosen with respect to the short term quota limit of $7, 400$ requests per minute.}
59
+
\item{Generated connection rate stared at 25 connections per second and by each test was gradually increased by factor of 10 up to reaching the rate of 145 connections per second. That allowed to make 12 tests for in each of test series.}
60
+
\item{Each of tests is identified using multiplication of generated connection rate and number of request per second as the horizontal axis labels. That values correspond to the theoretical request rate or the client-side request rate.}
61
+
\end{itemize}
62
+
63
+
First of techniques that was tested was a classical dynamic page which was using the system resources to determine the language and present internationalized content. For grater flexibility and minimising the repetitions of common parts author decided to use template inheritance. This in basic let to use template blocks in a way remaining the object oriented programming. However both of them cost the CPU time and in a case of high rated page seam not to be the most suitable solution. The most common approaches to this problem are presented by next two techniques: using cache or serving as static content. The most interesting part is to observe the differences in serving the same content using each of those techniques. One of parameters that has a great impact on the rest is the server-side connection rate that in Figures~\ref{fig:sm_benchmark_normal}, \ref{fig:sm_benchmark_cache} and \ref{fig:sm_benchmark_static} was presented using green lines. I case of dynamic page the average connection rate become saturated around 45 connection per second what gives the request rate of 135 requests per second.
\caption{Response rate statistics for dynamic page served using the Django framework.}
69
+
\label{fig:sm_benchmark_normal}
50
70
\end{figure}
71
+
This number is close to the mentioned before short time limit of $7, 400$ requests per minute. The test were done several times that way to as close as possible to the this quota limits watching to receive replies with the \texttt{200 OK} response code. Additionally it is also interesting the observe the distribution of the minimum, average and maximum response rate with increasing value of average request rate. In ideal case all of this four factors would fallow identical linear curve.
51
72
73
+
The second technique is cashing. It is a solution bundled into many web frameworks including Django which as noted in Section~\ref{sec:gae_general} is supported by GAE. Concept of this technique is based on storing data in some fast accessible space called cache and with the help of it repeating calls can shorting the request path. A simple usage of cashing system could look as presented in Listing~\ref{code:py_cache}. The function \texttt{get\_from\_cache} returns the data calculated by function called \texttt{some\_time\_consuming\_calculations} that next becomes located into control of \texttt{memcache}\footnote{Memcached is a distributed, memory based cashing system. This only one of possible caching backends such like file-cache or database based storage engines however it is mostly used for its speed and support for multi-machine work mode.} for a time defined by the \texttt{expire\_time} variable. In case a subsequent calls coming before one hour from the first calculation of \texttt{data} they wont call the \texttt{some\_time\_consuming\_calculations} function but return the value stored by the \texttt{key} name.
74
+
\lstset{language=Python,caption=Simple cache usage example in Python.,label=code:py_cache,
75
+
basicstyle=\scriptsize, % the size of the fonts that are used for the code
76
+
showspaces=false, % show spaces adding particular underscores
77
+
showstringspaces=false, % underline spaces within strings
78
+
showtabs=false, % show tabs within strings adding particular underscores
79
+
tabsize=2, % sets default tabsize to 2 spaces
80
+
captionpos=b, % sets the caption-position to bottom
81
+
breaklines=true, % sets automatic line breaking
82
+
breakatwhitespace=false, % sets if automatic breaks should only happen at whitespace
83
+
escapeinside={\%*}{*)} % if you want to add a comment within your code
84
+
}
85
+
\lstinputlisting{src/samples/py_cache.py}
86
+
When the \texttt{some\_time\_consuming\_calculations} is truly resource consuming this technique can bring efficient savings. In case test results for cached version of page the plot from Figure~\ref{fig:sm_benchmark_cache} does only slightly differ from the curves of the dynamic version from Figure~\ref{fig:sm_benchmark_normal}. Used HTTP headers shown in Listing~\ref{code:resp_headers} can help to avoid subsequent calls from a single user by using his browser-cache mechanisms.The test client was ignoring them as it was supposed to test the worst case scenario when all of the connections being opened come from different users that use browsers ignoring the cache specific headers including \texttt{Expires}, \texttt{Vary}, \texttt{Last\-Modified}, \texttt{ETag} and \texttt{Cache\-Control}.
87
+
\lstset{language=HTML,caption=Server response headers for cached content.,label=code:resp_headers,
88
+
basicstyle=\scriptsize, % the size of the fonts that are used for the code
89
+
showspaces=false, % show spaces adding particular underscores
90
+
showstringspaces=false, % underline spaces within strings
91
+
showtabs=false, % show tabs within strings adding particular underscores
92
+
tabsize=2, % sets default tabsize to 2 spaces
93
+
captionpos=b, % sets the caption-position to bottom
94
+
breaklines=true, % sets automatic line breaking
95
+
breakatwhitespace=false, % sets if automatic breaks should only happen at whitespace
96
+
escapeinside={\%*}{*)} % if you want to add a comment within your code
97
+
}
98
+
\lstinputlisting{src/samples/headers_cache.txt}
52
99
53
-
100
+
On the other hand it does not mean that that using caching did not bring anything. Because the requests to cache are normally much cheaper from the requests that involve the server-side operations the binding quota limits are lower and allow for $8,640,000$ API calls a day. Usage of \texttt{memcached} did help to reduce the CPU usage what is a big advantage of this technique. However it should be taken into account that usage of memory caching will consume additional memory, make the system little more complex and by itself isn't a fault tolerant storage. Besides one of the biggest problems regarding cache is expiring it content. In case some page component changed on some of pages the easiest way is to flush all the cache content. In case of application that profile has similar number of read as write operations cache will be much harder to implement. This solution suits best applications of high read rate.
\caption{Response rate statistics for cached page. Realised using the Google memcache API.}
106
+
\label{fig:sm_benchmark_cache}
107
+
\end{figure}
108
+
109
+
The last tested approach was to serve the entire page as a static content. It should be noted that the infrastructure used for this purpose differs strongly from the application server. It takes the advantage of storing stateless content and minimum server overhead. The implementation details belong to the Google company however the they exist Open Source projects like \texttt{lighttpd} or \texttt{nginx} which share the same idea. Those differences can be easy observed in Figure~\ref{fig:sm_benchmark_static}. Until reaching the level of 235 requests per second the response rate curve was fallowing the server-side request rate curve with low value variation not crossing the factor of $5 \%$ of average value. In this case the CPU usage rate was even lower than using caching and did not require any additional memory usage like in case of \texttt{memcached}. It should be requires much more work to be integrated with the application replacing it prior dynamic content. That task might be sometimes even impossible to be done result in bed final experience when done not wright. Thus it definitely is interesting option for applications that are stateless or use lots of different media files.
\caption{Response rate statistics for static page with use of static content server.}
115
+
\label{fig:sm_benchmark_static}
116
+
\end{figure}
117
+
118
+
An interesting comparison presented in Figure~\ref{fig:sm_resp_time_comapre} which collates various techniques used together by focusing on the average response time. It is one of the parameters that has a huge inpact on the user experience
\caption{Comparison of average response times among various cases including serving dynamic content, caching response in memory or making use of static content server.}
0 commit comments