From 005d279b34053764b39a725d959958da8b4350c4 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Fri, 7 Jun 2024 12:49:27 -0400 Subject: [PATCH 1/7] Add performance up to 2.14 blog Signed-off-by: Fanit Kolchina --- .../2024-06-07-opensearch-performance-2.14.md | 161 ++++++++++++++++++ .../performance-graph.png | Bin 0 -> 38923 bytes 2 files changed, 161 insertions(+) create mode 100644 _posts/2024-06-07-opensearch-performance-2.14.md create mode 100644 assets/media/blog-images/2024-06-07-opensearch-performance-2.14/performance-graph.png diff --git a/_posts/2024-06-07-opensearch-performance-2.14.md b/_posts/2024-06-07-opensearch-performance-2.14.md new file mode 100644 index 0000000000..db32ea9ee4 --- /dev/null +++ b/_posts/2024-06-07-opensearch-performance-2.14.md @@ -0,0 +1,161 @@ +--- +layout: post +title: "OpenSearch Project update: A look at performance progress through version 2.14" +authors: + - sisurab + - pallp + - macrakis +date: 2024-06-07 +categories: + - technical-posts + - community +meta_keywords: OpenSearch performance progress 2.14, OpenSearch roadmap +meta_description: Learn more about the strategic enhancements and performance features that OpenSearch has delivered up to version 2.14. +has_science_table: true +--- + +OpenSearch covers a broad range of functionality for applications involving document search, e-commerce search, log analytics, observability, and data analytics. All of these applications depend on a full-featured, scalable, reliable, and high-performance foundation. In the latest OpenSearch versions, we've added new features such as enhanced artificial intelligence and machine learning (AI/ML) semantic/vector search capabilities, neural search, hybrid search, flat objects, and zero-ETL integrations. As we continue to add new features, we are also improving scalability, reliability, and performance both for existing and for new features. These improvements allow us to support ever-larger data collections with high throughput, low latency, lower resource consumption, and thus lower cost. This blog post focuses on performance improvements in OpenSearch 2.12, 2.13, and 2.14. These fall into four large categories: text querying, vector storage and querying, ingestion and indexing, and storage efficiency. + +We are evaluating performance improvements using the [OpenSearch Big5 workload](https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/big5), which covers the types of queries common in search and log analytics, including text queries, sorting, term aggregations, range queries, and date histograms. This provides an objective and easy to replicate benchmark for performance work. + +## Performance improvements through 2.14 + +Since the inception of the OpenSearch Project, we have achieved significant speedups. + +The following graph shows the relative improvements by category of query as the 90th percentile latencies, with a baseline of OpenSearch 1.0. Every category here has improved considerably, some dramatically. Full numbers are available in the data appendix. + +The heavy green line summarizes the overall improvements as the geometric mean of the individual categories of improvement, showing continuous progress in performance. + +OpenSearch performance improvements up to 2.14{:style="width: 100%; max-width: 750px; height: auto;"} + +## Queries + +OpenSearch has made the following query improvements. + +### Text queries + +Text queries are the foundation of text search. When used for traditional document search, text search determines not just *whether* certain text is included in the document, but also where and how often. This is needed for the classic term frequency calculation (TF/IDF or BM25), which is the basis for result ranking, as well as for highlighting (snippeting) and proximity matching. + +However, many applications do not need this additional information---it’s enough to know which documents contain the terms. This applies when the application wants *all* documents with a given term, as is typical in an analytics application, or when it wants 100% recall to apply its own result reranking. + +For these applications, OpenSearch 2.12 introduced the [`match_only_text`](https://opensearch.org/docs/latest/field-types/supported-field-types/match-only-text) field. This field type dramatically reduces the space needed for indexes and speeds query execution, because there is no complicated scoring for relevance ranking. At the same time, it supports all standard text query types except for interval and span queries. + +**Using `match_only_text`, text queries are 47% faster in OpenSearch 2.12 than in 2.11 and 57% faster than OpenSearch 1.0**. + +### Term aggregations + +Term aggregations are an important tool in data analytics because they enable slicing and dicing data using multiple criteria. For example, you might want to group the brands, model years, and colors of the cars in your data. + +OpenSearch 2.13 speeds up term aggregations for global term aggregations in immutable collections such as log data. This is an important and common use case for analytics. OpenSearch gains this efficiency by using the term frequencies that Lucene precalculates. + +**Evaluating term aggregations on Big5 data shows speed improvements of a factor of 85 to 100.**. + +### Date histograms + +Date histograms are OpenSearch’s way of grouping data by date. Almost every Dashboard or Discover visualization in OpenSearch Dashboards depends on this functionality. For example, you might want to aggregate logs of every HTTP request to your site by week. **New optimizations for date histograms in OpenSearch 2.12 provide speed improvements ranging from 10 to 50 times on the Big5 benchmark**, in the common case where there are no sub-aggregations into range filters. + +### Multi-term queries and numeric fields + +Multi-term queries are commonly used in analytics to aggregate by many terms at the same time, often retrieving only the top *n* results. **OpenSearch 2.12 accelerates multi-term queries on keyword fields by over 40%** by taking advantage of Lucene's IndexOrDocValuesQuery. + +In version 2.14, we also use IndexOrDocValuesQuery to speed up search on numeric, IP, Boolean, and date fields, even when the fields are not indexed. This means that you can save storage space by not creating indexes for less commonly-used search fields. + +## Semantic vectors + +Vector search is the foundation of modern semantic/neural search. In semantic search, documents are embedded in a high-dimensional vector space, which preserves important semantic relations, then indexed for fast retrieval. At search time, queries are embedded in the same space, and the search engine finds similar document vectors using a nearest-neighbor algorithm (k-NN or approximate nearest neighbor (ANN) algorithms). The documents that are semantically close to the query are more likely to be relevant results. + +Document vectors can require large storage space, especially when documents are segmented to provide full coverage of the content. Decreasing the necessary storage directly lowers resource consumption and cost. + +### Vector search cost reduction + +**By reducing the vector elements from 32-bit to 16-bit (fp16) floating point numbers, OpenSearch 2.13 saves 45%-50% of the storage needed**. Our experimentation shows that this has little or no effect on the quality of the results: recall remains above 95%, and query latency is unaffected. + +OpenSearch 2.14 has also improved the memory usage of IVFPQ and HNSWPQ vector indexes by unifying the storage of index metadata. + +### Efficient filtering + +As a user of nearest-neighbor semantic search, you often want to combine vector similarity with filtering on other fields. Simple post-filtering is inefficient and produces poor results. Introduced in OpenSearch 2.9 and improved in OpenSearch 2.10, _efficient filtering_ filters *during* the nearest neighbor search. In OpenSearch 2.14, we enhanced the filtering logic to further reduce query latencies for filtered vector search by approximately 30% at P99, using the Faiss Engine. + +## Indexing + +So far, we’ve been discussing improvements to search latency and memory efficiency. But we have also improved indexing performance. Many workloads, notably log analytics workloads, spend most of their computing resources on the indexing phase. For these workloads, accelerating indexing performance can significantly reduce costs. In OpenSearch 2.7, we introduced segment replication, which improves performance up to 40% compared to document replication by eliminating redundant indexing. OpenSearch 2.11 introduced additional improvements to segment replication, allowing it to use a remote object store for improved durability and scalability. + +## Storage + +OpenSearch 2.13 enhanced storage and retrieval efficiency by introducing more efficient compression codecs. These codecs reduce disk space usage without compromising performance. Additionally, OpenSearch 2.13 introduced enhanced data tiering capabilities to improve data lifecycle management. This allows you to reduce storage costs and improve performance on a wide range of hardware. + +## Roadmap for 2024 + +Here are some improvements we have on the roadmap for the rest of 2024. + +### Core search engine + +OpenSearch continues to innovate on the core search engine. Our full plans are available in the open on [GitHub](https://github.com/orgs/opensearch-project/projects/153/views/1). + +We are planning the following enhancements: + +- Supporting fine-grained customer monitoring of query execution, allowing you to identify problematic queries, diagnose query execution bottlenecks and tune query performance. +- Optimizing common query execution infrastructure by developing multi-level caching (tiered caching); faster interconnections among components using an efficient binary format (Protobuf). +- Improving query execution itself, using document reordering to optimize query throughput and storage efficiency (graph bipartite partitioning); analyzing and rewriting complex queries to make them run more efficiently; dynamic pruning; count-only caching. +- Modularizing the query engine to make future improvements faster and easier. + +### Hybrid search + +Hybrid search combines lexical and semantic vector search to get the best of both worlds. Highly specific names--- like part numbers---are best found using lexical search, while broader queries are often best handled by semantic vector search. We have supported hybrid search since OpenSearch 2.10. + +We are planning to enhance hybrid search in the following ways: + +- By executing the lexical and the vector searches in parallel, we can achieve up to a 25% latency improvement. +- We will support re-sorting of the hybrid query results. +- We will support additional algorithms for combining the query results. In particular, reciprocal rank fusion has shown good results in some applications. +- We will return the raw scores of the subqueries, which are useful for debugging and for relevance tuning. + +### Vector search + +Semantic vector search is a powerful technique for finding semantic similiarities. However, it can be costly, because indexing vectors to make them efficient for search is time-consuming, and the vectors themselves can consume large amounts of memory. + +We are currently working on the following vector search improvements: + +- Accelerating vector indexing by 80% and reducing the memory required for indexing. +- Reducing the amount of space needed to store vectors by compressing vector components from 32 bits to 16, 8, and even 1 bit. Our experiments show modest reductions in search quality (recall) with 16 and 8 bit components; we are still analyzing the 1-bit case (binary quantization). +- Reducing cost by employing disk-based ANN algorithms that use external storage (SSDs, S3, and so on). + +Vector techniques have also been less flexible than lexical techniques in many ways, so we are: + +- Improving search relevancy by supporting reranking on compressed vectors. +- Supporting ANN vector search for the multi-tenant case, where each tenant has its own subcollection. This is especially valuable for customers each of whose user organizations has a large vector collection. + +### Performance benchmarking + +We are planning to update [the Big5 workload](https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/big5) mappings and settings. In particular, we'll add support for dynamic templates, configurable merge policies, match-only text, and setting the default query field as `message`. + +## Conclusion + +While we're continuing to expand OpenSearch functionality, we are also investing in improving the performance, efficiency, and robustness of the system for every workload. In particular, vector techniques are being widely adopted both for pure semantic search and for hybrid semantic and lexical search. + +The OpenSearch team at AWS works in collaboration with the larger OpenSearch community. Without its contributions to testing, feedback, and development, OpenSearch would not be where it is today. + +Stay tuned to our blog and GitHub for continuous updates and insights into our progress and future plans. + +## Appendix: Benchmark tests and results + +This section presents details on the performance benchmarks we used and the results we obtained. The data was collected using OpenSearch Benchmark to run the Big5 workload against different distributions and configurations of OpenSearch. In particular: + +- We used OpenSearch Benchmark---our standard benchmarking tool---for testing. Rally might produce different results. +- We ran each test against a single-node cluster to ensure better reproducibility and easier setup. +- The instance type was *c5.2xlarge*: 8 vCPU, 16 GB, an average instance type. Using oversized instances can conceal resource usage efficiency improvements. +- The Big5 index was comprised of a single shard, no replicas (run with `--workload-params="number_of_replicas:0"`). +- During the test, we ingested data using a single bulk indexing client to ensure that data is ingested in chronological order. +- We ran the tests in [benchmarking mode](https://opensearch.org/docs/latest/benchmark/user-guide/target-throughput/#benchmarking-mode) (`target-throughput` was disabled) so that the OpenSearch client sends requests to the OpenSearch cluster as quickly as possible. +- We configured the indexing Log Structured Merge (LSM) with LogByteSizeMergePolicy because this optimization was in place last year, prior to the blog post being published. +- The corpus size of the workload was 60 GB, 70M documents. The store size after ingestion was 15 GB for the primary shard. If overhead like doc values is eliminated, Resident Set Size (RSS) should be about 8 GB, which should match the JVM heap size for the instance. Having most data resident in memory should provide a good assessment of performance improvements. + +The following table presents the latency comparison. + + + +The following table presents benchmarking results by query category. + + + +If you want to run your own benchmarking tests, please share the results with us. As always, we welcome your feedback and contribution. diff --git a/assets/media/blog-images/2024-06-07-opensearch-performance-2.14/performance-graph.png b/assets/media/blog-images/2024-06-07-opensearch-performance-2.14/performance-graph.png new file mode 100644 index 0000000000000000000000000000000000000000..f3fee0baf369b5bf7af7fd7f9eb634e87862da81 GIT binary patch literal 38923 zcmdSBbzGEP8wLnCNC`uCcQ*o((k&nz5(9`*LxXg8cS|S|(lJO%m$WoUcT0n?&-lIX ze&7DTyT4r>e)9m&9L}6`pX*%rbzf&9)YarM(Miz}5D+jG6=XCK5RiHh5D=SDp8`K& z!`R^mo)Dci<)siRM#y)8Z-OoL6s=TM5m6Vt36j{8MzPUPW z=eQ5j#fc^+pDh~nw?FK+4`Oh&U_1XF$rh$_|K-z2{=x<5X&i+JV_zIq+)b-Bm+2zU z!zylWe%oa6t1vHiJ3Bk4y{Qt!QQzw>s=>0#wihduT<|_q&w-|%j}CAqZzdq1fwDT< zeC{^(O&&i`1)rTc-88q~i8P&bV%#0+i_O)(jl16xY)@$av(TP`Lno)_^TYY(@@R?8 zu)#LrmG4;+Pa{Qv$M3BlHPBTl5T?kgIZM*AgJp!c~O8yjD|CGO@%i;9Y59(MB{ zSk}C%MXx8)+UVtD$u%lBwzr2Y(~LjgCgLfk&Uf0Ko2{4gHk-u~e@aMeCI0Nbs`35vVXoDy z@&2;0JtfPbP5-F<(YI+Wl5In`{WhiL?x?x+=<&AlQInI0{Fv(Tx-ZbAoYL!n=i;Cs ziZhWy#7zMLflSN~1pb|jWQ3eGb=nq2L|g^ZfVDMq7FJeGwN;acGa?3;6{WU|X~V#H zRMdjOL8YdxOdEPOHf6WNd4`1syVtbDew^*Mdu1sf1IGdjZ@z_MSn-P0R5f# z`IBMtv+C=c&eq3>SKd-hvCPhE-4Qj8>v97_L$7I1Vg#4fXRJqy#V=TXs@=bP+%3-2 zv0NX$PGVF|jPp5wAoaa&v^(1vIJ^HnI&`wy^|SVLy^rk;Dw;orfvzqSq&He)bnH>V z2Q%dUcE556Mf@b7ZNX-`T!VZ}4SKt$+g>U|w>R7F*Y4z=#%cVbpCooFY-B_&;6&!b zPJWohMt_3Sf?Z=t73h9~$LrGe=J@f!p$BP195?ubs%p%l$9hcaUBFcb#oP(>XaGt` z^MNLrdXYMjmya)nn^f#W-TlK=d)nho-ebbnwV5REXIXlZ$o=%{BdZJa2^+(M`aJoSkGIjP?`MGLnUX>Mp$LojoeWk>8Bj8 zPt2Om2H8F}HR&7M=)P$uDgmRWefjc5eQrej&h}ym#HkK8(yt2j=;6Y8@#i$oPs=Cj zxz!5?7+o-TCoOd-A1|+#2^5SbMQ$hMb&7#3B{{~KgZ;E1$1M1@=|LQ&SSi(S1tfY& zBt<64Vxe6HeQLCLe?|lxG@SDZWv+p{uNQpgmZ-G-DS1kkO_F&Y!0RjhP3XLmT3uutA=>oFK&w$S4jT~3lA2005GMbn}pFn zU;MW~umEoGjC=c?v1=?9)Z%m;ZPWX1;g`#&>pf|ai08;3{Lc;HHQ@6|j#wrqFQ47{}jd#tE zW>g?W0tM%z<~^JjYg2|(3Wa_UKnfS+2$$xOvi`0us~4m;Z*2Siy=G%)D|q!;BQ9Ff z#qbQ2oo3ke!&(GWvRBO6%C%I-u7xjngcW!GNp&a`okA(#4%=z`7JXZb=`gpz+gVCN z0}FRB&|juMNDZpKS{3@)+6VQAOPY7%U`guc>26?6)Y~7isvchSh0^P#aOx zs1;l%3Mi3&vg3ZeB#@$1#PBk%xCs|BhOJzYB~3y3nK()vN0q3kE4mnh->ncO>b~)m zjR4kPo1SIuJlL4g7tx@`BBJd;gq9xZbX@^;K~m&o=E;(uO6gSl#)?rN5uL=Wq$D4( zdByu1gNR?-66vQZBhh20B0NmRDLO%;XdgQjJ8AhrM%RuH? zj+sW83MlFlo09DZqnfEa_Xc)1t#o8c!$WxmjVGL|XSB~rvR7p2u*j3{suoF&+=b3w ziCrX{`)`Na**?W^TmYq z&goPdWJHBPVrQmOYZCWe5<7kG-wCm=H+bJT5j??qd7lsVQ4s4hd3vH8FP!}8ST;8v zm?FtU)FKj$PgflQrnKf-vN+*j{2WnExxYd+4M7SGvJo}v(lsuOk>Hj(F0 z*>lfkE}f1w1n&p=QdMv)hn;|R!Shicr1u*pz#HFT9}n}851eJG0^iyj#k%kV~9nQ{)JB9+HYmk6+mXr{{~?>^jQ#^wCO3n+$WR z|3jXkVgTix0ppPU-5vh9)l?Ap;r}a5hJPzu0sw;9@8Ngers1#K)I9}&B#54mNlz>< zb1J8CrZ|s^>H1!jnJ79t*AC~38H(Spf`?K$3^zWdy-CdAdpk&=Yee%(;!oXtpYOx{ zO-0!Er-?vx{`&wWNy-$%8X9zBsU$f2Vk!3NlCrYcOl~LAHYW2P>VU@l;_Gc@lw@T? zy#MT*q{Wa4DnTHHqIU-o7l$%SbUu_;@VPzG{rnwZv76Zonzx0t_3E?THxXx>p0^;g!IMLl)r+S2` zq;tQNi=+GsI?;cd^C=e2`%b%Za&ji7rs%b`wVh6Th%n3($Vf>+YR*OYHeGZnU3dRy zD8VnV*#g2>@le2&$#5>Lw77;Ec3o1Oo9MpmDGzLu5(8IlS7Q1wXnrKe1HfTF$~Y-1 zM!2}RFr(aj&azL8z+)=ag9K~;gZTeWSiCkzEk@MhY!{|#G7r03YP0Dtes7ssi<#BMggUiXxflgp!?o>^U9 z7yIA!feXceci#RT^}@^lE)d{d{eN(c{QqMm|3A>ney9(|oFuM-K)}|v%*os%MM5@O zYkgeyFE4ifSYf<7iB|WUq<_~=$2x$KV&eX`zKN*hP3^j9l?%U`9egX0 z`DC0LPQT+6kSJ1lRbK!1b(_HJjKw8wb5%j}SFKEe$AYn=;Bt_8;{ji5C!rLP=jqG9 zpvCIrRxbnY_3Jkz&@@*mVc?R|*h)22{-2zLV1`wun; z;Qx(eb^Kxk{xWF1zp%Ja9f@V$5y407PB;2+i{J7XR<6flX~LSVR%vtOsR7R3|CLb^ z8qLG`e=s;+V_2bDs6AkO>azLgb*Rb+1?hN) zodv6^GkE@b!F126iCOX2BfxHkVTDa?2F&Z>_#>7&Jd@e?GZ^6#|L8fU)=1GTl$kir zun>gqJhxzJra>BUVkxIJrN+AH47czR^oyoJDJ0KFVm@BqzX@2#u7fo{Y<@#~7zJ$nPlCcriY%0-ZMTxmyoj^H03fF0YA@N_-M4&iu` z_S8pef|D;+hv{)uEO-rS&Zxi*=idluGBlDJ_hF8bek1o9}MqLQ(O1hgae6+VeuhzVA0r+lX$-xx(1kjM@+h zkWZ0*iT!5M)O$*98EmRfjdE((DGu*jM?A3p!eqLi;h=R?cQLvAF>OI`ANjP8VsnbI z8U>=UC2ZgOEbpuS1d9 zEvpu9wN5bt(3l9ZIAZyFw@C9^#NRN}@#m{Jqo<*a)+5Y#O8*W?4f4+SgmTr9FFG4K zeN``(bX(93Ii!J4yDkm@XFs8^DxQR3P$qgK<~~XnjvJH!AzU8T`eFl7HEQc#s_tDi zncDrDPi4_bXNHt<+%JaL7@N~WgBO;MJQG)$SWXgjx$v=+>;QrY#kPaPl_=2c%e85@ zrMa6Q`sfkq9!5avwho zk6hwGxyALGBetr-tZBg3b@1^KJkOWdYHWu9XtV1ab`eP{&HLR%{DcVGTMC(1N9 z+RlCqm|}xQgZBx%HIEbo*=k*R&S|%>hq&eYHr3Pbt(*?Unf#w2z7U*tR8PqXo{lZX z-`Su7oUzlj>$aD~ediPflChzwqi}xP(FlV5sC5XN-XRX=CO0czn!j&x>`Fer{jL}e zRtvrUStg!t(El1&A>1D;zk^Hn7wo;clx0a;*lmx~hId6!wvc7&UlJROZF(I{H)~g+ z)rS(cpzs8k7UU>rI?QmBKifMVIlt1$MY1%iJ^b*-e$~Sk*Pzxr)M164UVn#?4Cz_k zNtb#jW@r?_w}jAGzi3A~Ob*`*kRa-+wHY}4R5lQG+^=CeP)2M8j0^Smf8bneZgzB6 zwxz+Bu1`bqWR~()gU4Jh5OW7jG<>AxkI`8xa->$LWzy84xb4`aFQx*9vOQ0S@XDV< zF3=9~7p9%jEf&UVOyOXnz<}6`U!3Sh>-rK^J`y~Qw@KTLlGRg*d-p)q6+$H<^ZD~iSY-dOuu6?lUuv_ON+K2zq= zV4bxqN$26_=9kyB<2SB%M1hAwlR=_lbQlY$7sHgI#r|1bM40GgN)D$t2$=<(yG@p9IeJO2r+j*ok>Kmf-IW zTqUR=rhFaA#?~+*+ccLy*lk*22~{ zJ6`!S{p%qhCf;29QLmeun^Ub%%Mr!mEyTdUwgi1!J2a`gJ!vhitW0b5x*9BVEw+^3 zp5qlS)UdLmQ1=Onf+rg27S9m=ZUvxh$*J_U`FR6j-+LI%eC&mpL9^ReEiJA7wVtR= z8G&jlJirE2kvulNH^TQPod4CK%Uum=HA9kN9NCRgt`%Z z&1_%I&|=p?=5;;MVSML#^JH`HWf2)x-uyd#ef^;2(Gi`!=Hs>$(SGtwU@s{1yk%88 z**Jd}RL@dh=fBZ`f3(~|-g5RN0F7?a!cRG_F0MSeG{V?vtsCva0V@`}@0xm5#7JG8 zx==n&M}GTO{FTME=)mv``Yw@2TDk?wq}9W$m|tOD$WFQ+nz9FP{adoUzSdb9?-=pv z^TELtmMrP$SiJ%fkj3v=MfdL*Ln7)FI{(((Y-kT#eJbp-6W;pMb~6AGR0m+@{RD;Y?F>y z?tYx<6wc1SlB(1|H6DA(>`C_#M5tW5*jTNGRsUI8`{%hh`WAN-@E%Ir{dY|!>Z1;9 z(`7%6PY=b6>fX&T3rbb;#(Z7idnLw57Y$DH*e6r_j|j8~GRN(Ks!@B4zSu9X>f?d* zez`Ua*5(%wAbDIbNra1k@87>K%Fln<8%sgvF5P;$7J^5J$r9* zLPWwNpE2`E9K9(MV)qgqrHO+^#rg@2w`+O`VZSv>HYD7lHtGPTU=<#cV}g?k$T|SN z><*=InUIVahy#_M`!%BWczEWC?d`qnxy`aoM1y14Xs_LQJ;+w6MT2`wiUcbxly2(E zd%WLfU}BP=F4r$KHzDFQ`qIf6*y&|t7NSBT_WF{)h4au@nrkUU)Rn`_^7q39uPC3% zi8!#}B+pPcEQ^)xb-vjy_;oV#t8S^V6Ke@cF-1m3PRV0f8m$xgpl(%Ea9&_YY#B+i zLne40kO*&yGF~DtIMNl4UB4dlY#nMe!^6(bd-t4?sKvfpg^a438loMQv>+Q#6!;%G zM}e9h@jirmi(w8MJ>ZdNXLfwdCRocfB@(F8eNl3ZQOBTg!)ag;Z=0LdaX?+dtg)X7 z;5LS{FNV6+jQQ-Ekf=)3(+HnucJJ8L>&z65c5`!aPt;xu?`Vy=6jwf zAPrE*m)qslD@>aaZZ!x~TIG{T2b3!UzW^{_15F)Yz9;EEjG|JEhD+j+&U5YH4bjqj z%o}|MC)b~&%t-;H9FHJ?d^}k{Kl1ZaBi(BlQr`ZNsw)ESo(kl-+3x=}^6*TW)x4m0n4TL* zlY-aCbXAiR1N+O?o_SB+rs&gWb+M}l5)QZ?;5QmJeKGB_#-khf4mo66kEvac^V zc*~*X{EOFJZFV2OKHpT)&CfRWsPOhYl`!-PxdG^l6zUFkq{X)$8trpq1A=~!?>Iym zNKFpq9#TLQ8Uf@CD@JT_ZSt-`@zPUQxi8W3z4Fd4`#VLIuK_}wYIjxM4})>ab6*0; z?7FqE`I00N&~~uV*~ba@gPUcbFR3Akl(7}jvY#c@EIe*F0kJO(7)B9(535GliSnxw z9aGw614u|DN3Rk=bx}owA}{0b_e~#0dU(ssE!Zo0xzIqV0Bb_AVYHueBUU4r+{Ys~RfL1R2RnHuscZhUn(-#f50JO*gy<0tv+#<}%B7CI~lFgD> z=&Btm)TS~RS(73Z>O2gC@agBGI};;OS@68o!mj8=Sy^y#pGwV@a;38X*+s) zI0G!0wI3zbToi5qZCDZ=RT96L%cRZQ($Ucoy%a*QidwXZui*YV$Dk3mJ%u)q>F5cw z^|=atyQQpsBbvec;WK{&MZ)KXLZ8usoa{Yu<|L-$q~FEc3Ob`&bA~glTFo^2c8(QqA5UV^^^8h~R`^DjpG6P;Th9n1iY4WEF3>;z8LuN<40mf1`EJiE@SLMPoq2@JZS+*)ShE+mRWmFw$-t8~qyt|nF zL)!i-DEuz=)BAPI65@>4 ztdsKhLAG-m>>}?Cw2Y%g8iKPkycHzF)IZwT=Rwm@eGjP%pXruc^CSlu3AO7|ulLKq zG3Pnn$Je%ere!@qg!8X|%3m}$<}_M&3?YD%OsCQ^?LC9bJ$a2dieke}_5CQ`Z6C3- zad0+Q3RHoY5$LdW?py<(!HpZfsZ8o74KEv@`kUCeJHPT5LQ=?VED$U1e z+O8xGWx9P*mwr9=PW_~N8&qd_Fb$CU-b!POa*F`FFF)7}Oam#!QL7ac*`4T7^ribV zsCk`y9hjejcx7uCyXb&;Kmw!$PDBs4mJfS#UdVX|zbgoTbGlbg0V@T$`_y&YPal1v zcwk>np=Zw(B^eU^&DFXm@|9Z5CzfGlq+HsMOp7}tI033en%gi1)h@CDSVp&DvvdqgP3N;%&! zc*aWLp&OhI1{2atU?sgswMwwXs_nQlpf9E$Mk0`+rW25u zX%3YA{d>=;16-S=k|2tJ3Aec10?>F>3;m(byJWt^0_)AXa)@_rrkrYIr*Kv1NlwgQ z_49#*mA~f#wKGEMJs{vyCDuQG{f*`Bc7_U<6`bPP$K z>PK~dxU^iLkeWVwO{i4D-(5O@3Fy#TrT-G}`VXQs1mOa?n!H1LorhS)D!+I%7f&N! zsix=?(A}M#27?xr0Kk5hU*!;7ClrfsRqe8?dB+;3Q$si))F4VEzKE1hsNj<5C49n! zb{!0C(nBZ{_KmZPe>|jFQ-@(1hhN@uVr;*t3*4s&OI)odLft`g2U_#1>qOf?#vm-R zGGA6-s%1^YaSBk!2sL~rn>?!6Q4gYyoCIiHTMZ00TOaexsVfwdD}KaybTEUiS({k) zgGQ5`P?fO@fPCMl#|1D&*ndvi+j9KrUDdT1t)8*eIwB@XnGwMOb8Ul1<0BnColcv2 z_B_s}G7t?FA-VUV)YAA} zV!c+>vj+3picuTmR20mKqx@IsD+;tXb*JQ1kruj`Ok~nXRsb=vBIOjDv3GR;ZMLvv z9)qxU-H8Tpz{X+hZvsg29$$h*T|=3dTl`v`vQ|aI7=`+<`=Yp*ladaLk?ncJ_hXEI zpL}@%IdGYCg=u3E4}obLJfs|B@qVBh!B&L6J3WF%_P@Zdf<(DnAIm|hBcy}gey*itNO8!k;8bg|(o z(7-hnY&Tl%j3sl__2}wJTo%sH?uV!)W0+(41i;#m@o>Qlc)*Ud&)|8ktAt*&I3@b4 z0a=YF^_?`atbsvhG?m11vyXtt!1|+!p9*ib1?t8mM=Q?&E9D%1QuI~)dG%E!T(-8Q z2db=OtBkeQ&bFr+bUz#V2*YfejcEwJC`YYB9s4Se-)YbgW+xNbYk7>SA-xDLShYyg zz+_sS8+cJ?iwaq%~*jcx8j^XJoz zUd_jj`Wg+xnY@}h-cIj-iRHq57ci@8V~fJHGDsATlH+EIZfShvj>Gi0EWi>S;*dHq-BS&QSWMbBp%&Z-p^W=WJY9K-9tqfn8|FXjrRnR zx3?eO+1sZA$&o`JVP6oKvt~}QcaSh#x%b7us(Ip--&<1{a?Xawm)|)Ec!0#$_mrx- zM?tS~wq$k<+JAsmBAN4N&Ew=lU$h98AZtq+JJKJ#Bbb$AO}YU`ad~+;SHmnpmZzQ_ zjN+yR{6aw!s860(SAdP{4*3F~oJp^zQ-^|B0y1b;4Iuk!NHTP1WCoDr_UmF2gc z-Sbo$U?&l*k+MUi;HdPqvRnt}%y==J!J$~>5BogM+I37eIs6aiK@*#=`LY5j2Ix=2#fG7{&gRrfqdGV4Y!2Q zJ}*cHiM<&jZ|!viumOb0JL{P>x!C(nq}W&YqB|OCi@(yu(F}NT*Tb_OdqI4XP7bV$ zKpR4)u(zV0Iymg#zb`N(2aC28%JQ1W|5zooN_a*zcR(*qzaAu$us9ST;iC-9S(kLy z8?9v`!^I^nCVf6mcRL4763u*?^!3;)VztvZ>reWKM|i|N)VB2|*T&HU5(8B{R}Tr_ z?QaB&u(c#+pj7u@ck4wI)YVN%WO>r#X_i~c;ts&_-g&RkOH((B{-UyfnLy5Fxpf84 zL%^6iu+Q%sVa0!f_syfE7qYAskf&wj6rK)7Fnky5RRcEM*bvg+p>+Osx$YG91Zpr0 z`MzZ@>>dSZRCPWIB+72(t1K_$UPg|E%6yPAxgveu^W1f5rn%6UW(hzYN+a9K_R}1| zgfj*|=3TuxI1)Jj_!X&9nK5`=EgTV%kFthmv|j}cz+ENyY}A$ zkpnm9(=`!0hKmB^!+{>F+l zkHgBff1dhQOOU7{3GMuHvxp!x8Loy*$wb9bW_w(Wp3W6@0nv3P-RxeAy!^#?r)V#UC*MsZ`lf?bW43+6nEzfL2={DU4!H=6|rxqW{HxQT6d1QV_Cd=YuTb(DP+G@ zOWNApsKq0H(R}91aY?Twj%n#t7X{?`{GL;Q;THk3~@lsJY_zNb{YkV70?~NMVyS1m7rx zr0Dluib{-g_Ppdte>MBQ5~bqV{0jE`JfzUeLl~gynDcDy$q?`^vY~?U%B2<9y3yC> z-+D@DM?NTYJOXt)pXa?_<9rQYL}QAzG~mkCnL%F^(Hbn6`+N@J2ZZ67y*J_EY9Sr5 zvPOk5__H6W3#6HaRx8rvOZ_sJ%vE4PHp73DxCnrRs|Dv`{1RyH@tpCN=VkigXvPL{ zNFt7A2cz|7@6Ta8w=xx>o{FB8+ZdzEKaH zlm)QjtQOyB#)@%4CQyU!qRk;8UvOu}Bl@uY>BJ}HoOd7_z*v5Q2`u(s&n*4q@An9= zuGFXId(~|3&Fr6yqTq!yR^{N%Na*VbP;Wq>W$W#QOp83hw7#JcZ6!-;OU`ZLcV=LJvN1gm>iucgT)>Rh!|7Gw*vE$r@bDpXsxMG;Z= zMIIo<2AW{9>U)d1rUw2<8vF*+m=g6vLWt+{Zlb?FOG(lH<=Yc?aA)rQNti@(Y%Equ zNl8LvBzn_!uGdD^ulvt{zi1Hd)uNM+O|FLYv6{62hH1dxLo<_5CnM;YG2w)+WmbG; zm&Cu$ZqvItz_w_~fH^vxhQY+H**s72z9iwYLPwdS+-8nVWdBX-Yq@tM=Y2ZNU7Gz1X~ws7x;?sGIA# zMREbgdvFBKmm~tVDY&`6Q|pJJ8=e9eYucqQWg?%vn{sdWZ{!(YwRLp1qEhc%N%|T7 zgQ_WQRO^i;fGbCn>C+UR;Vy8_YknF|=0z&O?)9A8t=Wf`@n;|K-li}*GhRMOW!9+G z&|W-}-b)k%mTo0wWd;d}JS`xDRiL&)!tNodZV9PED+11QlzoD(po0OI^HLWMtp`ws z+M_BCqw>`Qyj_+*OSO65j*tUosJ20(T@=zF@M}PW9=T87IP#Do&3&X29v7hXL~e!^ z4eSyPVy3*_7!9LL{x&U^YA`?LruFbAq_V%R6Js-Q0<8FQnpydnCrJ=>@lw#!=?V|1 zEU8AZ8`kST{E|mcnm-xDSqkMQoI6#EI#~AMEwQZSwgH(AQnvyJM&LCu8!jFqK$<*8 zqsHxg&kr)yBkTzSQ>8vZ!Rm)Qygm27Hf}^P<;KHHGJp>TUrVSLZYWEUIO%xev>qdf z9^3fKx4HaH>jwo5a+@w6iA2R6Q}^AsMRV9ab6$CA<7vjn`?G86M%#HJGf;m)RGkw4 zo%+PmEWx4i5g&NbP)?y&imj46P99YycZv*hf4Pd)bhSHD)JsehbC{Th1^DbvGXfm} ztPeN_Z8egMxT#dBkPVaC|2gQOu2RB1K|~zl=@AwdJ_8KzB@u43=6KbeuZz{RrEcJ+ zwZnzRp^F(4E}&)rKM%v~PHaX9>(vwnPEDL4ayqyoR`+zF_bJU7j-oOCvK)N~Z+sDN zY6D1D9_uG;y)_P%WR8OnV$FmVV%Vmuh%^hfRKV<--Q6qVB}>#0JV-25%-eLN=+mbG zKnu&Ue!36H&?nr8GHm}^-3*R;0dM-065b12BrfxMwELjoU>N89=UBh=c;JxK6uqeH zcPbVp?4lIrg6}bJQz8LU0Kr{mZ#udcDIoaqmXg+VrWD_(cJ_uV000{f{VG$$CG+jD zU=C`;S2*FG7ii~LWh&O+%e=}UF^-t>bN7q)D%;yUuAsmpLYs-xa?OAU54}(3OBE=) zO{`?Q&}ZxXE`PYop|IozzC0nQwc41{d+p;f#$ zN7iEt;13AGmwOF>!+4qeday_=kII9RL`DK-|8qs*N5y2|sBFT7|vLV+AQopVlh$mJ$NVBYyRi=&Mw2InTs{#bG*HR&hYv^@synL1;&ZM!lk{&|%Xsg~S0lIFf?d4(F9xw_y?E zdJF)U#osWw0+|3_SO65Lsk0O!c;dk|NHsu4Wv=kcJ za=lGeL-M5tl&Ua|=xQ)DDxnIu2gtbtOd`ygy|IhKh1;N=v`w~y0#9UWejXNvl}by@ zAPKunEVD?5uS)|;*POdbDEISMu=UKI%b1_)43N$ow; zM}#a{-c*+$^=Y*lV_K30wXj|~vSu_kLE(qolOGD8M4D)g=QHfFGD7uNtAv6A0^=NN z9B$Kc|GGew)q-|`bcPP(^hF_5vVt|QxRc`m7nlmNE71{;jg4oJSxxHbgeqED(K1c$ zU|RO+G zd_k_kYV=QfmLMvMu_in^t>_gl@9R4XS$xM8Ve+Dt-%zx3mR!p5K!#|BU2P54t(m-h zOJdLS{M^*mk^MY94HO7eMTGR$DHuHbk%mQHs92n~l*yw>i#j}&t}T!Z)h%o8j%4&4d^rq2ZU z(-}ObU+GuefYoYGm5*OHdH`hCKNY8=hhD3*m7y{q1ds!IVqEmK1&)^>&T3KX*f^>m zMI1L}P9DSlv#^W>@vyg+(!( ztL>5@Y$YchkJSFtmtYikiXgyud$*wFVZx?YHMGA)sOy9V2uHA zB@p3HH5oN{r&I9J@L+^1h;X14@A9eId@W>GP;EZyUMyUjxRAa=pFHDjX>JEc?~xn4 zBiVmDB8mUjqin$Tt1&3l#P@G)a-!a$%vif>ksjCI5U9m@$CEDXS&mjyygyva49!t2 zS4?JW1u`LR##q2x!~4l16<)+1!-CDdR2JkAH*~!TxH#?yvWBUf0t}@t^K_mD_j*7~ zNNSe#m>u-sV+{bv>|C>Z<@_$^dpU7Q3L|)b?Xi zi+XAA$ZKM?y12QV$o>Yj9Hv3uKZ0z? z6sS~X+e(NPam)Qc&F_$-R37<+mT;d?^@cX}ZQ0ka+SNUa!(A@~fpTs;aH`?*4z+Jq zBHXlf$au5_&nSXc&e#4&g|S63GQQVVaFfl ze3)-XdA_SP_gKRXO|_Wjhd@oB0uDWqW>8(a9l~ZC(F5xk!hw>^y08<>V8e9xc@YaE zy6@wcAJ7SUxmq2Kq(XpTUBRWYwiWy~BoMX4y)`@hga+RQxFWTEf+hrCCpSD$V+^1% zdj(sEczt@V49712!!L)C@)hr9c_irzG1J+1v=eM)iC-~~tY-E!_+@?7R@c&++9ssS zMw)@wDZVoTsm8%++^kovVYawcrYSDvYba{EEU!+ZLQjzxtlx@>#O@?hl2VB{i)j&f zXYTh`phn{6#c>X|K94E_`W1R$xiRfF!|Y)_dX?AbZFap>A3~}Dv6`(p>vIna!KDma*zfZf%-sF(IQ*E+qR*PF91D_@ev9iA_*2{4+iy*e4!j)HROpz5Z_Z-g zm(M8|B=%F7cL*1T=9NF$5M?A?(#9F^n=MML9zD~|50B<%cy&HK`l7)DICaWWl9mED zy(9s0kw{+;!Pf#q_)z=M)xqH-K$<|2IE~N>?fI$D*eng;Dn)keRT;tY=kA>zO^50l z7_cth&NWq={-9ss2i{Mye8=Vmz(sBlnyNgl63EXLy$pztOKehFd*v&Y65g`Ugknej z>08((6uGAeW$u*`f%O31BOr8EPfaiW9TTsQz?di)JBanonuDX$$g->n2h!^V?JYvI zQbzGa+~+5gF0qr!8w%W@bAqVrzggv2Uq&w-Q%9#6sWWmNyh~^pv4^-f8?mtTJFSl) ze^v#OGjE=ywT@>ZB1%X>0KR~wI?3`uRc9dcs8RAc4idc4Z6*ACe-FD1f5QI!q|4B^ zn`C&qXNheT0;Ep^qJxR8i*bW_ra3~XL%DtHzbRU&P_SS!HAgc76I=Y&dGv|84EDLKklZaw81o#1`7scag zXiR;zH5&*>Sf^@ip^@%Ki|;lGwg8v%2@bFTs4zb-8I$PfaeB^_YyGSfzqD&EgWS+# z6Cv7?nrjmu3r{!m${GGvkccd45HlgZ50u?G%mqGDA+s`&;bVC$hvgma?s?gfac8%ffkm3DLycdTo?T|c;y z7!8iN4T#)P^oT_82wnZ?k*^4is(ownH4|RIZymE};OPLFO0?X2~yFhO#VD-MC5GJb2vaiN?8lV~;fe?J^ z>fFXI+cfST=PX5h6{cRt(nw5E$Q}F6BPn-|+)$-aXx9Tc9}bHJAbLNv+Ypl%r~((B zOwS_M*JJlKntVvx53UA$+A!P2r3pb%{9Yesa^dlz+V3pA@Bfba10ahX{Hj(}938gj zVOsnI5@rKR;f?4J3aJ1z_bnMS+s_H}&aj>jeBcV11cOUJ!P8U)FzjcykDkfpLEQOQ znU{^TVNVVwIh!(*%m2c zVc`4GXeaWk;W1dlEps?SkNab+{%g6?eF}kNQbVbBHozRB&;wiETzqP=A*+dIVQC5I z2LOL(4K@b^Cq<6D5WvzV^a^fskt&|OaXgU)_WC<>;wN)xhs5BYH-JMSxxTmWXi8$0 zvqho(-RYc9ERS2OVf!cDs)ePBk7<>Hzdst1pF1Gk9lu3#iToq_jpc}JElQ+OLa|Y( zXHnmH4YWf13Sf2(j*mT$()3V0LnC3T9B`fhA5o|WzejES%B*!)c5_0zS$hQ7la^Fn z69v|ts4f$Y&)5P0HdcYtYf8=%EcLsXdzxO;OB66!qUWcJ`+RuMW@QI3k_OyxeS9im zCz*%e5|6KQy^e$jhKKRHwe9BX6n%lCl)&-*Goq66^5maoxSot`-5qBK*>CG5NLvTcfkxB<_X2}ER)#y^Fs<7z~DE$T{aa*R}UrYtJ?3TnEIufkzK- z&DMWPLVMTxq9j_ML)Z%6W&tXzpP=jVBa#R{m-8MUG=5r=>k+`$>|j%ZwmKD$Ya# zZQ^y=@*qDakgmD|P3>4AmZq&o7$==4qL; zoXVNp|M53)T(h8~AH?l=$sXxBh4@h_M}JQWi=i8bE_Lfm%$_X4uNDtnIaNG54b6)= z+oX-=Q2X~TKLl~KI8Qxfh3@WP z8|mwxB7M^GaCWZeSBl=RB<+$tsc4qrMS{Tpdk6uW3|)ACN3LrXFC0Kcs-5I&Ci!oI zg?D}$5*v13=5U1dcYYlYxsmB5%LpYcd?!@H$t9zPjKnEPnma+0Vn(}j&g8YCGXLXc zcr0yc*;6b4Nwxy=T957;7NgGbg7BQOq@IRbrCD6M3Dj|FMVZS1LAp{++Ul#Nx?MGC zE?xm3J(2J{Mr8xox=b3c+KaDBw9ACB=7?jm=SG6AB{aGS8UzT7VuH`1b+%t9RV~XA zTjBW)I?$)8c0*Mubwi+&_{n&&!yl}>Fxb@{Go~22s)x-}7hvo`zZ&O?5!ULESH&T~ zz9InQZGeJM7f!cdBy*%EJuo(M=w!KU!LSPkd0uFHC4Zic!s7Bt-RFQ<>SeqK*R`N* z&aSdTH9~G2yAa_(%8Qpg)#TQLtLrufCX~w%c0knI?HY0JT*IXo7HOS z6Zt{7cCSQR|IOlt5Q&3mO+p{D#S$PWu)AlS%isk`Ar;JNds}?SnY>Edo)fh}LRL)tr&~!^lBp;~ShP0Z3Y3>#7aR5=sz1 zpMY6tT=9QS!4AHPbYYdM%!jX82imZhfk;Y84(Am2H=Mp=VbAVw`S% z6<^PGqIy_0kbDuMa~mr!_o7gHK#6WOhx3xJ3A1ytDU#+hs(UaL^jPL$L=V|-D<{NGz-u-_R?S{DD2DMTX#`4|uqyP_#8jOMglCMOQCs=E(DGP*Y&?e5nCATnM9F2-Om`*GEt;%Tbw6 zzVQUgwT5IgTdE*i8h6QVld$n&6 z*aUCDjn-nv{%IC4!}zHTXm-6ij_vI1BsSwt-o-@#2il%L?GrY?dn~HR zMZ2a=-2c415duI$NFP0xOsfm%IQgXgApqxkoBSLxzyt5EZ9{hB*|na#$Fc!@3kI&K zG>`7?kiJMz#v+RVrU+Dtz-sXIYp=pWhBu?dpd;z00ElnyTk$YvKr03UaGJyA?hMN? z$}vQ+3O+j@2F(h6$9)w9kVR?#o>X1HNi{TQmN;M=ZwII$0Df7l2;wt<7e`o8pVGDQ z*_v7VGBg5V&N3z_VYS%Mg)E;etY&yT@}BbWeNQ^^dN*MDDwA*x@CY~lf@OUY(Og(4 z5z;|FmCoBL`*y9;Xmmtx40MWG$ z=?*PK@1RV^n%6~KeH2Q>re{_hgRP%PLUWfAX1yzcp)n8aF9KlaN2~BeSS%6slq+lK z@+o3P+?v*98ty6)jzCnk|Mx)rG_%Zha8)Q4pJNNp{~P928tA=oy)~c3=AuXkFc+tk zC)OTe=Ww4el0N7e>dWwIm-Z9UEclJBRHY+*K3;&Gh1=E}(k(SvCGd8Tpx+TQNA`SJ zJ{|QgKIiAWlKT+o+A%qd-aXypBV8%_yMqRXS$&n~luWnxr}Dg99omzB*;XrO!&J|2 zM!X2XP>hlOl2o=hy#mu2f-;u`*P4obgYTd#5oabWX8X65lc%MH{AbsNyc zNJyg$DTYpnyWZd`z}L(*DmV!>nhaS*teY^VQUrK~D^*eB1$r(>gMi0hEMTEnN35;B@G>wzqT$?aVDoL=ETWR2Z0u*zjh%UU+M&!gO|seha2J{sbY(IO&WWOx)7j z$=ttQc1>^jn6SY6$`7RSsPN?%6%nO)!2&%Y6-Z->w}rfz@b#U5wC9eF*s+R7@})Qu za1ORnGCgUlT?SXJ^fHM4?MVE=^Y_u8AGfgn2jl{cO1REB$_QPs(mp&(ndlU2-Nm73 z2FGu25ka0Vz)sC_Xvo#cOLFy>snO(?+mPMx<*{Ac!b>*s6_^73t~R2D)x!|Sct{F) zD=0_Ehn@W?J?(=4`hR)SB>dax3FaXIJmT8UZ&x)S4EW|Civr88Rmx>pWZ3my*j-5? z>))Jqi~{svjVUOg!8mktmcGD1T-UxKI)jvfCIyAys<}3nCrk6)SOWvf+Q$+#LR5Y~ zYtC197t1)HrvhPx*6nPzv13|+C|EX5(R%$2v-Q}_F8CYyYC9ZyfGD0N&ncg1$9HsO zK0??BUapDaYGo5!4Xz%#!Y&*Jc0=qZ@v*=?WL?oUOo!n;Xs?^dHoiT4NO_hyA}pv= zDEW*ZFn0EBI1u-Map7X`c%1it`yHSa6CsbmDG z(e_>UdLhzlJCw`j2p~&q*=$Cnda}6N-E5Pc%%qZi#9IQj-o!rCVQRl;82U@o$qX07Q+tA&SKdfC7w2YhOQ@>H~NVtC)Edfi|Qai-Deg#}-x|q+uLH z3I0H|5^wkzX~>6!CDBh80d@yhlVx%?evZLlE$<4O8Ym$XTk{*)U=kfOu2Qlu96(W! zzllC3mwUT{&z8fhEmxfmFtmd{X-mTR;9*APh9#~UCYqftRHy{ZR|m*3`+)0Bsu}7I zwO1=JPoxZ0Ef8Uy#guAn`Dg-EG8l{_2v@~9Ine-pb4gO%%85JksVfwWpxn`aC! z{A1Fy$e`jRzb~9L6Lf@DR)f6^pSlz7%Q2HILnPO~-aj^!QEY)jbu%oc2e0oGB632z z0GOHo({S0ESU6t`J&mTqV79Wbw&_W-fCj5QoOZh6qN1}x+P5eZjwv7EUI%~FY!J== z8Z<0H{=yt}u)yz2K6;MCW5x?gAksHtn2y3jWU1zSqB`)B6G7FSJc;E=1dP|&d@0r~ zOgDmqNLMA3OoxMc;S+aUA=lqsfZE$T^vp>|y9{7!2iuQB{bKLGA`d%?yxI<={3v{& zVS;ty7!Rix)WqZ9Wyv)U6^e$IV>YfFTCLD@KZ=6VeNnW1f1VvQN`LbcVOF&Vfs>Sg2MZW|+D_ z0lmF*@-9UNG0p22`ton0?<)i9W~3Ply4ax;v@oBVCzxO}vzd)z60Z4I7rig*89hEy z1L6kAPEmSYzt)$83LMP8%~T{}K1sO!LvUpMB8*ns3BzR5*l(%%0ly@-@@U6us`#lFPS9SUG#Ybfw!aGm8p#&5--6 z1%dh|#mWdCry>HFzH~4IL3gLgv~OmovO&*TDZFVGW(U}p3SFcFzGl(dKmmOWn1V8; zg)%NK4S(%%*^P!MGru!8ZRlyRgX9&A0OXx!pz-AaFR%2J#rN?vj(E2>@2BCA;6ii>izYOYn)F? zeXjUJ!>s5IoS*e;?sI{5UmmcdJaxf|uryv!ChM<{7{Cl0Em@BM-LR+DdW{ljwt*QN zHO%}BaLF_lT~B1W04CW79pc)Yo_ho+BL^7ZBl&K=*mv3oUsaKM9YPF42BSx>6348v z1mk2LQ2neOWs2)SUuqT4!!1W+H%UiF*j1|aILrYQ!12IgR;&4XqNMNoY-a<6-EDR= z)so__vb;QT;5SDN0AHB%rnHk&U7>oF6fk+EIN2Ny4?^EqYZpNXM3*?Ai@PnirL4G@ zSQ0>{dL>2w>ZQ}~!Q?!Cn0fS^MT6||$;n2m*CP-D zZlAc^olW%v*wmmS0HcoxP_Bv#e*CzcGe7PQqY@*HV0B1RFS#6uTzK_VHxyQIUn2pP zeSdL#kxfRKFnE5k72+@OlU)X)Q|+0hT~tgxx+2&alk#E)o;kHi^ZW(lS3xiI6@V1k zKs%48KyQW;T>>FMS)!o0cpu=;AxLo4Gk;2v8Utc!PdzFTQ}MWLvaiu_hgrCo+1au3 zWGExp5E6`^*5GLf;=2F&{SC{aZb>BoNV1bRpBzn4uvbMk_to+tas%I}aRovP;(KF3t|AyxdQ3`qLRCofrY!aP3( z0{wAy^;`QF6+i@t?o}&d?Ri;RKun#XlC^wnMocX7cHu38Y6V^<7U&PJkE(>9P56u; zBCqii7IO>Hl4B}odU6_qZ6(13JK-Pen=p|BfJ1}it~yw~D>uTE2a(hc0m-6_OA*wx zwVQ+%Ws3to4fAzIn$wsMpdLLRRxPK|L44M7t88Cmx-`im?-UI2mIb=SR>oYG^DgMm zT^yHhon}->j{czjgmwinhWdCq#q-p6TEnf=H&X-giU!kOs(y_%v_GR+-9CilK${up zHt|mREh*DILGfGP*}K_~=>zKkO|OvzERHjxcCb`LTVnz)UjB!|V4uG^eyhJ!oMEEc zVd@cfm(Cx`hup<$ClG*pnNz5u1U+E^*zwS3%)Y);2;bzKofzRa5;W5H2zX5($LR#e z(T02*mE30!&?VB4(Mi#)W?-BKHJ=aPz1JVN`>G{HH&HWdsOoaU$v{!iqphaca6^*v1MRR= zD3C=6{ds#M3EC^btn&pk3KUkCTfBT;-DwUJo@UqT+PV}W!_p?E8PT9V8bDGd!0FJU zj4wp(ODQOZpPDv4TGFV{m{;nr3V!t9N9xHBL^tH~Nu6$qGBgRUP_LuFIF_sZO#&4K zJaM0xR`!2D94f1wO^D#J(qB>^YYVd1e~SCiUI+i2lPa1=6HtdZNEU|RUp1|wT9$0> zxB=isVR3eWZydlb4tV7+kn1~#Lf}o$&;YFW5JKgm8`*kDe+{hztUipF6_33I8OX5V z-s$Y#P9cDmwh?AgcCz*p$1xc!otp0OqHrI$g;masqgX+77Wvu=OpRx>*j+;&Z%&Uv z>l=_b1c=SNpfK(uNQK7`a6jeXh_%n&!0+K`AZ3n_{?k>>(OzSfgq%uYCs>Ksa7rqu z$&kX9ht2eiNi8T>6U#bpnIc*R5uDpnmGV?uiePx-D>wSxvcCsC+;cL^F4+18uJZ(B z+LZ8_w7=s?OCjZ;gL}OQs!ts=>(4Dub{!Gn!Fl>y2{DS(%R9`r%X%*DxUBiHe0*48 z5x7G(uiJFSKE)V@LZ(d)Zif3)Z)uRvNn2xSy>IwF1ChZNTBb4atyKWeUpNuh?+86* z*AYXm1;ZEwz?4bofaQ5%n#

5RGtqX=uFAr zNnnz%(a+`dWmq5qLIzf zoN@uKjHO`vJmNXst)&*M0V?ZA5VsQmLMM!O5l?t;1^`1)p;rW+6$3E!9R@wuZ!~TY ze*hXZ?IG%E!PJvhTex(>wdq5x_`Su`paBL!ytpl9_M5CFS41q7C6g9k(r;HiN8z&eu_ zb!)j|&Ibv2HX!l?P7w`IM8u{97n7s}LM2E&VX|YgoA8?cC|fKJ1+ZWNiuc|qK-YBu zPX;-qN3Gn$h8j1J%Nmq9bHyK9rD7y8s-{W&Hg zB0SN%%qfP+sX#kz|M#~yDB6MF6JQEszzsb689dEp3517rhr51V)`l_$kIC3|0jCzZ zGg~rRMEhyx!7c?-hcIx`=lyjVW{Cu%)ZR$B9*DnW*yx?)Fc$=&6sqU0&%;=$mz#8a z#X6EJqY1akIjSevf3Y7fAP{UOm8J32BD`zjdc0*%El6ItQa=oA?KyV6IV=@AlPCY|w0fc3RDUl32>5<4bNG15`er$jhma@%l5-j+Yf=C1xAGORO zI8Q#kxC~~m|Ct;n@4NLM1M)tT`oRwX(~-?+sFO3ln?IV32AJa9srd`#NG|Iz{lVs> z=%efW!;WD{Sn5_3GyiS1deYv$yj)dNK>&({*&F^Y3^_~C|Ipa((C6!T8{S8-wu9sX z)A88^m(4BA*V&ru^uCj9?T0uVyp>@FT@*C3xil#Rt03V{*6r>uFR**Q6CzD+V0#@1 zR1K@v)XFwnPxv;6^1YwH3D7g!(#aDP)4r|GdhAf`@;lM!7n?b7Tb7+DEhEM}dmf-vy8<*eqjD}AN|Lj zVHqE(LzFa0u&dY+V0^JFFRUXsq%9~jglR|K(#)?)$P6IhJMBKJYOU`{zzc{#DJcpK zISb-2pB1U$9K#K~cmyj1M%ssmXp#jfy<{#pXR_7OqG2Wu|M)A;*S)A$qnx9PV14rG zSrw4Etp;Ie!vOsepcPG6f?$>g!aXjR{j65F1_2b}Sn&0iUXWoce>V0CX>{nKF!c&? zo^Na?UeFCe$3l7rc$K^fNI)k&hNM7BLqvySmjI4{3)t^;UIgOwIw0#;iU#Y~An6yX zJ*&a%2EGMWJ4OZCguhQ<#@S0~tUOyFjP_U%U{uo(!%|AJFm2%}3Icxbz7iv8L?n9n z?vu#Eh1IdK_pkvEN^_S!5N|o~4Y!uy&X*LyOT$);HjrTThS;W&)T>Dbxs30uOLn#%(D(|N**5* z>lJKuO;S`d-{i)*vd6hsn9gw2cQ!xtmHV)QDkKa8uOeFT2e>_XNt`$j_nMmhQxNkSVZ>C2lg!wV9j=tiJm6K$1n(Vadmx2~>!6GLsvUWu?u}(c z_fWqZFisc&&oV4ijdYEuLAi>hp(mdvDO#om&o|-!o?s1x97(^vyP3$Qn>+WfO0b!X zC0M@!!lrJB)0>G<*WCOGraBXn0ff-Yu1HI#bWrPDbVu-!dXa%!iWV4vk4f2z9t44J zzNp9PY3iObhei(eH=ESd{`etv2t@W7Aibl`22`{F^A0b<%MwpUIJ|c{D9WScTraTB zyVhmm(kLkO2M$5)b^eS-+8(AjiAO_E7DsTv%n+=OwkYib4a2QBz>9$W9K9#BsH>ZE zmM*Ho`B>j^$m%;GI}crd^Ng@hPL$v=SX4CWbkdaY^5CWC>!o!+MuA`(YyUkvgCQ>d zCTIegf1f4{=6yA+N-?8KlNboO1&TJsM(49H-;0amqM|TicIBjC&`-I^IQ2}ajt_SW zpu}uMiW5hIAu(6GMgsJI{`|4|ub7d9tX!UQnFH(tcY@Jhr{ew!R&82e9Q`MCaeC!s zr%}wYN+esQA-Xu7F6Xd00WO^>g@|OU5`3#06yRth%yKK1rLegk@ekQ6k-Y@KAvCw8m$GGrO1Q1uIi)* zCeu2;nuKL;ZHqCZ?6E$C^*{V%7xc;~KcfIXXNaoQS5mPy-6hbYvsv%K00WUyz!0RK zN|W&zJzleE7JD$>sVEMbNgC$IauVg)fcRN<4V?58(-pj6faOpB{>D1ffs`+@l0vzf zH}{w@$5z0eBWLfj>HnP9g+PW18g2Rf#y~C58zw5&(-n6|@Jswc$zNkx-CQ!%{QJb@ zuV+moA&z#;UI`+s^<@?d+;sHxSY}9SA3ji^|CHc72W-)DgZ^yL+k=_LZ{U>tA4Z{a zMSjN#JR+9?PC>b>i{Laa%Fn4aHq1kvcxEYezPLG|h_b%lI69_u&TmpuJHUC&EW$)A zsgV>nzNoPq236O;*pIIM_4C~tg#s;>{%FE*Z=hnRP5zm=l7#(0&iV~QBntn3_Qix=a?__bkUOVo>N~_VSHl^Wc|V*`8un^ zBE%1`AwXBC(>sGa$TNskR4518vE^jI9@Tb|?_zO$pKn8;WvR)a7F3~AK!9aA0ooK;}6uI#$c#NT5vkb zU?O|>T^gTDwf)}ggw`#X168Hd{uO_YeVBJ#HCQvAcIfrW*!akf3t~#hOvjA!o zzlm|_H0pLDUmOydvc)lLTL}B3R_&CuEFK!|9c2jld};9v3-Uya;f04BN8K=0PE>r7 z?G1+5AcvJN9-+1uLoaVvKf8#N?ULaXiN{DsH#zsJLMa zu2{?Pcmp}64to3%A|ePG!^oE=Jt5UD5X#FzVS2!%%z)y=f6nZrobk9$GzxH-F^GSn z{RJaB+uN}d$%rQV%=5F5Bd%#SW@o25n^Zhv44s!9g4*QT@hhtq-**QLOnWw+>M*+4 z1@(78e&eKKzYoPRb1D$cVNb0pfXij6qrkLa_D|vzZk>pyxRCa4K9X_ zl+-H)z7UgVf;p3P=6csQ@Wm&}Iz?0Kbt{{x67lo&tlxs6R_^Vo?FUjAL+*8Av?*R0 z7n}`06e=GmgMzD#V^}cq84xggHdGcD)eE2QOEs8n4kjDvb%%vk;TT1;ca-^7EEYz} zF3{QG1+KosQobc8MZm|35FEl&Q+#qbsmqv2+qc?FtGK8>Ut_7gRx&cAk?MrhEFf8_04A$FcaqnN_S+I*4TdfdCt8Kt|;tC$Cqs@YTjL@{d7q^RV?s(tTgYg1J5NizPuKV{Gd zD`RnpNXAD*3DS5mNfRVdlb8=~@6|#kSFf`hifyyn5>N7&Q&LnfZUa&Jj-*tJS+j>( z=1j)jcG{FCyQzZu>m6gcF!G-v{DH95SuTESd1Jb?kW{?X^c$)>_qvRE$_jFeiOgp; zWi^0eO;y-g$Y`h`HM#0q8h|x)DXXpU$#N2}x!d5y-PdP5*cI~P+fo`{r zs-L+2{EW%&3^b6-pEdK3IjM}s_u^let%XC0FD(CV8=tf^%z%@s^$b#P_h&Wg=dA9h ziQgB&Waga_Pg;~Ys2JVCKPv#E?VC=ObyE+TJNffn{k!<0*J9O@-dO=%g!3|e)W0BjV-^5rLLR~jg%&<-zl{mrloP0)Aud;TmHMe zEz|%GE7)IXs9Bm)hQ$$)3}Q$r!`fKjMlJ0tHHMp~`=>ZTwMmNV?u}l$0|~7Adc?d| z{t9teUhio#p6*FCL1Nu8(7nVc(6veknc7Je0}ST-rc8;q+RF!C8d@rzYYYi7V3CxB zpTun#8k}EqK6s|aDJZPw?4kc`^zweXpDUBNI*A|e5EkQ%kxxb{UG~KrF~Ae?H9CA@ zzqKyc!bZO=!wnSYWSPjrCP$jI5(l?GFI_r5_x-15kkhnPUE0SvLg1=bk$iP$TKI=b z)WUqQf;Qe?vjl3t%Y@=|$A0;9b;ODW);jC&B5WISg+557YgUZ3ya_HyLj(v6sfQ(@ zCmL+H%@`Ee{S58PmQ@qHyPHC!an_yJI+%Zh;W?G^>>R-6XQ#{b7Bpz3cxa@W5PzYM z%+Y+r`g_kLkbEsW@}(IlGBEmTt(w?wtab4pzPeL9vCLc+YA{;TD1_pCfTP-XsU?&A6iTy|j);u`AGk{X>KTvQkp;>@M0T@Vdq)D?1BfIsM>}WogMtf)lKtK zCrW_#A@*kXbn+ADLFRjrb9HDTu7CTr?S< zPa3zRKdp_8{u zb2#+5x$6@GySUEBq&i!d3=S!4KGd1PT`xlYZLbO)wpNFb!grufn1^j#UqZWe^rngI7-rtZT3#Nx#DBUR~%3RG0)O%@#8n&6Fx#Xqams z1qWvRY|2_qf{1k5rYMw1XLon{r2BGX2hHVRjb?LWkIe)buO7YQmJ{*sHVFMHb}pC2 z>EeJ)hX9Md5fomt#I+|o@$J%w>UATXscQnCqijA=&L>$e)<@dNbucrf9p36*YLP_B zaaUk$E0!Is2C9x91@oYJ)Hxz#2K;{3av)k!BX)Clsruz26fC8~IAae@a_(4)EY$-# zIfE=d40PCSdP1%+#}mBPp`4=qh-$*khG$=W2Jx(ff;`m`ysvc|-w^XP*I}(I_87BV zd0vmphuU2hvI2X+h zJr6e^C@!Oc*C9T>a!D1@zg?Lund!LK{#2EA0)7?ywH)b_JRDwZ&$tFEWZ_X`b}B@` zRtl&Juo2wqtE{IcjMCY=Noac3ZrQ@QJjhldPHuYPSeN#GgB#8>0f$@qS;ePUYu3@I z9);EBZpSL)S(yD)X7=Pi1|O%eTODb%TVWkS0KYej;nI20VK0W*$Iy>J+rQJzf%t&}URyW(;sU2K9<`?`a zSl(f+^cHG1*LJr*UQ{(_^i&|MH&BojwsL=uDbD_liU^3l(zIPIH%{-d^RV=ZCBGwy zr|;X$J4b3(X#Fm~+;s{5Q2vr8#cjqm%hwi}Y40_H*T3lqN};J959+DEOXwwnYJkVX z$kP`-?I;~~_;N-{XUfyHR)5;8EX=10_%uT&IGme|@dUhdvf3;=tOu*RkoWGgjym?D z!PEw)qN}D!X6Loa^3NJTe&X{`+%@<5ktB zwv*EGSe&>FLK&o2*7M;%pDvOykJN#*$>-uRcdc0w2XfI%mY?Oj@fMf%LvJR3C+QiQ zlHfhZ?_!)9;ogi;Si7I}aR}|#R{8IH(>_9QulovWj!jKdYE~_tw5SQ@zt^2F zfneQM7qOF~Hh$|N?DCOv$2gZN=$l9-Sjqq1YEe13qXF%0TnJB2_AK*MkA14GRJK7Q zg`#GcUqNBuKljWN@gtL0mGb84P6~beP>t>4>D;GdjDJ7o?|SPaCsou`+bEicWx{`d zI?y9%!@S6u|8pPz|NgM&k5Gd_f80#I;x{ueWpJ`YTYv|+^q7Mo!slQl@Lctrg{jLy zQw`tMdh}$YlPv+K)#rhY4XXe9Cg3ZD%9GMaAAJkw|I*}qR*u3{+u9XQVwvuCo(Kl$ z4Shu;v0!e#%oqIc|S$uXnUd{ z+8bBr78v==9d(mfl6c&f$tJM;tcM<0aLilzfU$CO0T>j8!1Tbpo=w(qHo)$0w1vfk zzdvl05#z^!VMILK@B?o!+O!%R?05XZC}w(%O5;dt*H%w>z+~XisI}7hDmnh=zmFF7 zj}aEw*Es;ABpd)1HTOZD&mh^N{=5D5uU_S1O%vE;?>jK`6WW$Qg)N)>)*hiwr#!ph zzYp;50yBgp0Wnz-xZK4=Kja}pla3!s1%;x)I79JNFB)TY7wo5G_bV#1_;-SQjL-hN zGyY!sOW#)T*`{1-L6TIuQz*LUPIpPT9q#ExDgU=hz=vSdd!l_nJn4^tRW<{9cUU7Y{bpi1iod@M=7#D)`r74lRBK1X@C@qGA!$p)75&C!q_k4xc<-lRl$%rN`@!4Ps*SPGL zTUPTEVNdxMZTG8j8&DOU(0s|=T1((7I~I@`Qd2dPhHy-`%K$O1-aF6_=2rB{Ku z+RRSRSO*|6j*e$T0CQV|*!*I~7ZfO`jxZsqGaP)W;c<7im;WuS9I(6QKr~6W*)x7n zo4TU&s*1+&ay^e^Zanlh%_DI;jXbJXCVY40izQcDRI@mXAIT<_A}1%EF=8HuauQh$ zM#RkHKXN*=;NlKkWsr*XN)mGitEc{F(jLtIZ`pj}M{LEHqaXF>dvi2;Ex_T-ykhXJ z^!^P=hcHbUm<>PscFrvN_FaZRvAH`EX?FF3gT_UUXa|<}Hdss6%5)IycP1x;9P*#A ztvj{B!1e{Mm78h+Pn&>c|4pZJd#T0axLA;_)O}qBPMWvGDLYi>s`H0$#)fI4YSR7P zfd`LtSqr+kSx*GnWSyh-z_CJp@Dfm?zsTPLoZD_X90gIhxW{ASoLq3H~h_1h@OE&0<--09ut`z1K^Ai_CV($=vooG>W@AV%hY+ ztN;5<GYh5GvFC`MRpEu<2FaCr%wNcB1QKnN>Iu7(z3w$1@Vu zTFz%phi1ditN{jO@dqUCnrXGyon3@w+PQD9drz>F55 zs|hW+-7E_9dh?1#xK6m7e2Z{9gK?Rw!OZ$!MK1)lGJG_-FvF_1=O}f9<~%{d2tDif zV0y*eC37P&JEx?gwE-DRG)hY}^Ei!+0aLtq1f0{(*=ZsU_}P*WAgulRs%$+kRj+mG z=8?cNV|CIA#F9=_Z9ZL(mj9i$MIMsrYQFr~H`MPkJ6f(h*|K@2D+GUynD}!l`Hp-U zHv5*%jb%9BxkRq_&tz1%?s{=wcC@&|&qSslh6Ns6YjPa{>#mpOI?@B-X0Ik4t6tz{ zW;kUcVh&o!+`JK|u^T1LWbMw2_=hpDki%frc3hUDt|gK}d^QB1eO?4n@#}O@1?i^1 zMl3!>je1#|?WSK?0{l&3#gL7u9~~Ao9UvRJ+4Q0)yFl1HR4>$)S{5AUW|kiAp$%Zr z)NWZfv8_4uTj*Pxz34;5mOjeVB~7DQakkOyU8{C~G-&k8Ymp`KMUWTh?r6}uyHeAa zG(Q@efn|-4Rd&BdJJrwBFwhlBU?k6b@D$Xf2b@0~;MJ<_;9XzojWX)?mkg3Un>6xp;czZKKaqyp zAuGJY#M8}Vm_dtI$!>VHF;L{%A`4BsJrRZ*ior+w=G@JHpP4Rzv+&g%Ecw?mDcY}5 z_l)F7)7|uctEdEMBCZ#Nhe$u#3%#!ZwUo90M*JhMug10;-22wus#%==eR&!GTOH)c z!zmVml2bdZ&+C5^Y#69&&lea`XYuOpfF?1!W;kAIk0?ADvp5J$%XLI>VN70uQ3`> zTwWXG6aH^uvMwQ*#VPKI%=Et*{~BDnbj!2YNB_RuixA26a>f6ABVPMAp7e_Ma2nrS zGf?2SX$i7wRDfJ*y8eW~j1-$nWI5d9_FE_s&m`>G!YTs$-RV(o=)R^YY8-=_?(0jS zrq52f09#-dqz^;b$n_%CvbO4J-QbwDjmnelf|A)#jz#0O?(iDW^P8UpNngF<0#tl+ zeX;aKpt@q32btRPHa4r_WQb=ED9Ty-Zmu@dIktrXWe_DQ76kdB=L`UZWRSSEj)Wzy zdWXF)K+0GJYPMcjJEC@6sx6$Y9!Su&<}4N)>&zREx}4-MRJ7d;eg%T^cuLFPJrgh{ z@U{KVKlbgdekHId1%9m^)G`U5LA_cg`*wyh?kCtE=D->=4M?OJ&N?f39wUZhU^zVUo zP&I6pU9sFJx3h`!HWX&OJB~f)>(f6DgKws$D~9<^LG5L$L)&2NdPqJ1?#ymiKgheo zu{y7tBX$hpwG2U?&#N*XUDeQmamUribFZo>atY$`zX4J&5r>@L9~aP=g`EVd&n7ke z7q3%k5MV{Oh;^FkTjSTDzHGpP!(IlCuse_=aS}H$)<)JgH0_D3&zHICq9XOGKF)Js zTssTKTw5&E+e;(s-4eU(vffOTxEp{>;!i-yW52!5eV-#8&&?|F^jEPK&nE6CpPAQF z=^)0#LcgTq5Wh9{w+0Zb8Cj7ko6cwZ?Hkj^2NKys1Ic`5o|3pO8?FxYV9>}=CRnkk zlGg(owjN>-s3H4jYb+1A<6#a^wuQco$k#|ZQQgx+?CvcN#9o9j-y$k zZ^3Ljz(KaL_j8TV*8cr$JBEvhLw6ahC@UY_`Fv9=A6ZZt|Tu&@)X2l53fy!CtL zeCA)0tIsbIT`x`9NX5oqJeS9}cH7wCtgF~=78|o!*yoKDh$ zTH6q9Z@2lOEYyLVCb$2=qLM&_Xb|Y z#+T6!$IscRpZyZuX4IR8Ww(^A>-vZ2^xF>De8D-paak>w@UEv=)bX1swqeil**Zdv zSc1AXuSNt$(4G@=jvR%2&5_S!^jl-lIr=7G3JS6vqpi&?J!64ie#87X0~_NtKdlJe z{^Z0*t+Ac;g@3Q4-+s2i0d^5oc9z^e(7FDg`Q}>R6mT34<8wH}1qs+*am&B#N9EaF z<`(CMa%;Ri1}BU)29jqIc8kw!QslqoJrd$wMt8FlvwLTkF)(CO`r3G?GfAquY4Gh_ z?GSn66Yq5}#@rvP^|jec-q33>CN8fc=rv{V^`~H7zPr=$$pRJTsg-#!j=uPWYQe79 z=gD&$EFYsZr>^IxhX#To#;i`H;o-gq;t#<Bc9k3jmi*pWc7*o zm1hrqvgxHNE{;|%_PE-JZ>SWwJ#3!@in6Sf##W5Un8!YJsjrpAwqKsb@NoZPWstm` zP)5U0E6rrsdiwc*NO=`B$LoYryl`1tGvxXFDGYA&1nznRV~h!X&&(n4@(rYJqbH@O zis3y^8-quGM)$&<`8dm@&$M`W7zz3RY`H&Up0n;IT3C2-If8o&jy?8gA-o@OD#-|~ z6~(LsDapQUX*8Wh_R_jN@{Y5bFFUgS3)Ae|B@7Peub?c(rM857+ z?0S`>t!3Qa)ecOCe?LtQ@Tl&PZ%+KkN-dCz*Y2=juM+Q(?f^S8-B&xgXT4{zh78@! zCBMD88(u>-*qAc7i*uMOfnF-BTI7u+{9!PZx0- z;muv5)F%~NoVZO-Hb?M@Yya%=TnlD)oxT}J^@d5W8EAww15%4aK6bLuO*Q1vMU71^3fYwfR#BB z);8t}hDfGf#=~g1`?^J-jZ3su!1GmaR+)CUwrkU40#o?}rYGKquG@IJUht3t%rHB$ z-R6s?1F9KT7FNL(|A34zT{o`n{BQG+ukc|ugS}0xBW6eJT%wVfj;XSP4rv>W5=-jN(lc8v-w!y)I9xBS zpdq!c*LXcM0w7}SflA21Jn?P#XOWMi{swErsm=wA>Txa20k6@Qs@BA()p#Bz&T2|U zd83r6e`m+vww`rh$=jVh#bKd(SR6e!L1WRqLd8P3K49UejtjBukX&lFY5yR{)p-=$ zCEiF@P+KY!m$k+aebkO7Fn=bbnU3@vHPT~5=qcA4Hja55pB0nx1`BG)dv{H--D}p^ zB~A$Mp1-M!xRVXzzI*RRdh4uu+(Rure^)V)I<<)PJ&WC+QaLkyy8yd^08hOzuCBqP zKdM_YTMV@ZzK1`z@wpk^sYc+zp_4yr3sAQhCi{ek_=nbN)O4ivsD1QvS^b!VcnIFg zt$c}9yJO*V&ZVoike{!%54*1zHUHRQK+Nhc+`n4vbzC6{T0LPv9HE~lBMHp86ntIm17`VY?U&o9qU&+~bIp6ByB zGNj^m!h;4lR1CLnL~>fr+e+vnIAm@;H2IDDdma85-}B!+|2vam zT3nJZKZ!8Sf1ZogiOUW>hQCsHnp;?#CY_pK2@)8_XaI`7FKZr}x%sx|NK)J~XdgNt zX3>Cw8(!ZXal<+zmW&rgJ~s>RQ|572T8}$U#JAV(fJJ_`N^^RSb22e@pUeRpe9C(X z(J;ym_jpK@lp6{>6B|s%?v_&T_AJ$ZWCn<=>!(c}*lo1*L(NbN)g4Dey&;1ntMw z=#cX!Z!0p=(;-xaTbnOphzX3@)QqRk)Y?K3@hP^+w*60KQ%ykT(6uCQ>RTe-`8;o) zX-CQQteTkfAXQOBbU3FjF(bS002}7WdszE8y%pTl zc{=x<%z!v5L|raR9Z6xZ$CO`L{-lU#!+b6Dc@Xwc4N{?Q^um*5m7CJ71_R+y`t15W z^swJnrbbL`bl*(N`RIwwLp9+EGv$0{>bmx`x2?4G42M6fa+jAM&C7MU*b6lKj1!}W zy9Na_o*OTno=DRW^)~_nlU;6%V%~k)O2Z5%{#C+hsjG|5$WcGF#n-m61p?g@!Br8p zesY~vCtRP4V)J~o#l;tf<6fPDS>CipSqPT^Jk;2*uji2u?+FoAcn6i}Qq8UQqv7NX z&|sFCXQOm!arsC*$8Pv4M-|;i->{|0ts&a9IVsCVMou&&7kJK@3}Os_F$I#8=-N0B zRfv_Xq=k--oddzwB#7udYfWnW6aIXibb7xQcwfR|N9yaiKL`%WO~Tj_ROMv@I2sA| zOBOcmLLd{+NJ~ixWx>XPSlFT8Ymt1o(Nd>*yj?AFG*HCi;B?x&*OnT_R{2L9nE)dv z>K8dtvAS$cqCI&OR2r*Tv8`1)4DnmH%-D5TYjZf&F*Cu|_@(L>j$iG^SBC~jvb#mlRuWly!gy$c4W8W)#}&*QOZJPplH+%4ipLfBf<5A*o%?eaYn+)*j5l_u z*Uv%~>TFf$L6qLFei4&CnQj=Bil3cq@0-h7PAG{Ok6?>4naJJ+S6ESieLJkA)={8R z2`_Rdl&Ik;E)QLYj+@ysf~BG`Oj)o+%o>Fi%PMpAb6DD3;2|y4x;h}gNadI(t)zsF z=f)74>v2U|R|de)z-Jd_ahmY_>Oz4u1mDTCJGr)GP zpF4xV!$L9S5DVlr@loEka5Fux#bFn5os>)!_HNJFad9zztYYsZ$V`dRIR~E0msJ#- z@LwO8m>f;!R{CQHE8(n;N-w5iSP`~2Kyd8 z?%6S1PhYvK-peX!e`^p}`G7tZ&<|3yM9@W*DqVw9 za<-M{i|bvk>%`p$w~C4@qyBY3zqwPECaR3_d*(O7%|m`b6mIJ`VE~$8jNSI Date: Fri, 7 Jun 2024 15:16:40 -0400 Subject: [PATCH 2/7] Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- .../2024-06-07-opensearch-performance-2.14.md | 88 +++++++++---------- 1 file changed, 44 insertions(+), 44 deletions(-) diff --git a/_posts/2024-06-07-opensearch-performance-2.14.md b/_posts/2024-06-07-opensearch-performance-2.14.md index db32ea9ee4..64f58c3f0d 100644 --- a/_posts/2024-06-07-opensearch-performance-2.14.md +++ b/_posts/2024-06-07-opensearch-performance-2.14.md @@ -14,15 +14,15 @@ meta_description: Learn more about the strategic enhancements and performance fe has_science_table: true --- -OpenSearch covers a broad range of functionality for applications involving document search, e-commerce search, log analytics, observability, and data analytics. All of these applications depend on a full-featured, scalable, reliable, and high-performance foundation. In the latest OpenSearch versions, we've added new features such as enhanced artificial intelligence and machine learning (AI/ML) semantic/vector search capabilities, neural search, hybrid search, flat objects, and zero-ETL integrations. As we continue to add new features, we are also improving scalability, reliability, and performance both for existing and for new features. These improvements allow us to support ever-larger data collections with high throughput, low latency, lower resource consumption, and thus lower cost. This blog post focuses on performance improvements in OpenSearch 2.12, 2.13, and 2.14. These fall into four large categories: text querying, vector storage and querying, ingestion and indexing, and storage efficiency. +OpenSearch covers a broad range of functionality for applications involving document search, e-commerce search, log analytics, observability, and data analytics. All of these applications depend on a full-featured, scalable, reliable, and high-performance foundation. In the latest OpenSearch versions, we've added new features such as enhanced artificial intelligence and machine learning (AI/ML) semantic/vector search capabilities, neural search, hybrid search, flat objects, and zero-ETL integrations. As we continue to add new features, we are also improving scalability, reliability, and performance both for existing and for new features. These improvements allow us to support ever-growing data collections with high throughput, low latency, lower resource consumption, and, thus, lower costs. This blog post focuses on performance improvements in OpenSearch 2.12, 2.13, and 2.14. These fall into four broad categories: text querying, vector storage and querying, ingestion and indexing, and storage efficiency. -We are evaluating performance improvements using the [OpenSearch Big5 workload](https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/big5), which covers the types of queries common in search and log analytics, including text queries, sorting, term aggregations, range queries, and date histograms. This provides an objective and easy to replicate benchmark for performance work. +We evaluated performance improvements using the [OpenSearch Big5 workload](https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/big5), which covers the types of queries common in search and log analytics, including text queries, sorting, term aggregations, range queries, and date histograms. This provides an objective and easy-to-replicate benchmark for performance work. ## Performance improvements through 2.14 Since the inception of the OpenSearch Project, we have achieved significant speedups. -The following graph shows the relative improvements by category of query as the 90th percentile latencies, with a baseline of OpenSearch 1.0. Every category here has improved considerably, some dramatically. Full numbers are available in the data appendix. +The following graph shows the relative improvements by query category as the 90th percentile latencies, with a baseline of OpenSearch 1.0. Every category improved considerably, some dramatically. Full numbers are available in the data appendix. The heavy green line summarizes the overall improvements as the geometric mean of the individual categories of improvement, showing continuous progress in performance. @@ -34,100 +34,100 @@ OpenSearch has made the following query improvements. ### Text queries -Text queries are the foundation of text search. When used for traditional document search, text search determines not just *whether* certain text is included in the document, but also where and how often. This is needed for the classic term frequency calculation (TF/IDF or BM25), which is the basis for result ranking, as well as for highlighting (snippeting) and proximity matching. +Text queries are the foundation of text search. When used for traditional document search, text search determines not just *whether* certain text is included in the document but also where and how often. This is needed for the classic term frequency calculation (TF/IDF or BM25), which is the basis for result ranking, as well as for highlighting (snippeting) and proximity matching. -However, many applications do not need this additional information---it’s enough to know which documents contain the terms. This applies when the application wants *all* documents with a given term, as is typical in an analytics application, or when it wants 100% recall to apply its own result reranking. +However, many applications do not need this additional information---it's enough to know which documents contain the terms. This applies when the application wants *all* documents with a given term, as is typical in an analytics application, or when it wants 100% recall in order to apply its own result reranking. -For these applications, OpenSearch 2.12 introduced the [`match_only_text`](https://opensearch.org/docs/latest/field-types/supported-field-types/match-only-text) field. This field type dramatically reduces the space needed for indexes and speeds query execution, because there is no complicated scoring for relevance ranking. At the same time, it supports all standard text query types except for interval and span queries. +For these applications, OpenSearch 2.12 introduced the [`match_only_text`](https://opensearch.org/docs/latest/field-types/supported-field-types/match-only-text) field. This field type dramatically reduces the space needed for indexes and speeds up query execution because there is no complicated scoring for relevance ranking. At the same time, it supports all standard text query types, except for interval and span queries. -**Using `match_only_text`, text queries are 47% faster in OpenSearch 2.12 than in 2.11 and 57% faster than OpenSearch 1.0**. +**Using `match_only_text`, text queries are 47% faster in OpenSearch 2.12 than in 2.11 and 57% faster than in OpenSearch 1.0**. ### Term aggregations -Term aggregations are an important tool in data analytics because they enable slicing and dicing data using multiple criteria. For example, you might want to group the brands, model years, and colors of the cars in your data. +Term aggregations are an important tool in data analytics because they allow you to slice and dice data using multiple criteria. For example, you might want to group the brands, model years, and colors of a collection of cars in your data. -OpenSearch 2.13 speeds up term aggregations for global term aggregations in immutable collections such as log data. This is an important and common use case for analytics. OpenSearch gains this efficiency by using the term frequencies that Lucene precalculates. +OpenSearch 2.13 speeds up term aggregations for global term aggregations in immutable collections such as log data. This is an important and common analytics use case. OpenSearch gains this efficiency by using the term frequencies that Lucene precalculates. -**Evaluating term aggregations on Big5 data shows speed improvements of a factor of 85 to 100.**. +**Evaluating term aggregations on Big5 data shows speed improvements of a factor of 85 to 100**. ### Date histograms -Date histograms are OpenSearch’s way of grouping data by date. Almost every Dashboard or Discover visualization in OpenSearch Dashboards depends on this functionality. For example, you might want to aggregate logs of every HTTP request to your site by week. **New optimizations for date histograms in OpenSearch 2.12 provide speed improvements ranging from 10 to 50 times on the Big5 benchmark**, in the common case where there are no sub-aggregations into range filters. +Date histograms are OpenSearch’s way of grouping data by date. Almost every Dashboard or Discover visualization in OpenSearch Dashboards depends on this functionality. For example, you might want to aggregate the logs of every HTTP request sent to your site by week. **Date histogram optimizations in OpenSearch 2.12 provide speed improvements ranging from 10 to 50 times on the Big5 benchmark**, in cases where there are no sub-aggregations into range filters. ### Multi-term queries and numeric fields -Multi-term queries are commonly used in analytics to aggregate by many terms at the same time, often retrieving only the top *n* results. **OpenSearch 2.12 accelerates multi-term queries on keyword fields by over 40%** by taking advantage of Lucene's IndexOrDocValuesQuery. +Multi-term queries are commonly used in analytics to simultaneously aggregate by many terms, often retrieving only the top *n* results. **OpenSearch 2.12 accelerates multi-term queries on keyword fields by over 40%** by taking advantage of Lucene's IndexOrDocValuesQuery. -In version 2.14, we also use IndexOrDocValuesQuery to speed up search on numeric, IP, Boolean, and date fields, even when the fields are not indexed. This means that you can save storage space by not creating indexes for less commonly-used search fields. +In version 2.14, we also use the IndexOrDocValuesQuery to increase search speed on numeric, IP, Boolean, and date fields, even when the fields are not indexed. This means that you can save storage space by not creating indexes for less commonly used search fields. ## Semantic vectors -Vector search is the foundation of modern semantic/neural search. In semantic search, documents are embedded in a high-dimensional vector space, which preserves important semantic relations, then indexed for fast retrieval. At search time, queries are embedded in the same space, and the search engine finds similar document vectors using a nearest-neighbor algorithm (k-NN or approximate nearest neighbor (ANN) algorithms). The documents that are semantically close to the query are more likely to be relevant results. +Vector search is the foundation of modern semantic/neural search. With semantic search, documents are embedded in a high-dimensional vector space, which preserves important semantic relations, and then indexed for fast retrieval. At search time, queries are embedded in the same space, and the search engine finds similar document vectors by using a nearest-neighbor algorithm (k-NN or approximate nearest neighbor [ANN]). The documents that are semantically close to the query are more likely to represent relevant results. -Document vectors can require large storage space, especially when documents are segmented to provide full coverage of the content. Decreasing the necessary storage directly lowers resource consumption and cost. +Document vectors can require a lot of storage space, especially when documents are segmented to provide full content coverage. Decreasing the amount of necessary storage directly lowers resource consumption and costs. ### Vector search cost reduction -**By reducing the vector elements from 32-bit to 16-bit (fp16) floating point numbers, OpenSearch 2.13 saves 45%-50% of the storage needed**. Our experimentation shows that this has little or no effect on the quality of the results: recall remains above 95%, and query latency is unaffected. +**By reducing the vector elements from 32-bit to 16-bit (fp16) floating-point numbers, OpenSearch 2.13 reduces the amount of storage required by 45--50%**. Our experiments show that this has little or no effect on the quality of the results: Recall remains above 95%, and query latency is unaffected. -OpenSearch 2.14 has also improved the memory usage of IVFPQ and HNSWPQ vector indexes by unifying the storage of index metadata. +OpenSearch 2.14 also improves the memory usage of IVFPQ and HNSWPQ vector indexes by unifying the storage of index metadata. ### Efficient filtering -As a user of nearest-neighbor semantic search, you often want to combine vector similarity with filtering on other fields. Simple post-filtering is inefficient and produces poor results. Introduced in OpenSearch 2.9 and improved in OpenSearch 2.10, _efficient filtering_ filters *during* the nearest neighbor search. In OpenSearch 2.14, we enhanced the filtering logic to further reduce query latencies for filtered vector search by approximately 30% at P99, using the Faiss Engine. +As a user of nearest-neighbor semantic search, you often want to combine vector similarity with filtering on other fields. Simple post-filtering is inefficient and produces poor results. Introduced in OpenSearch 2.9 and improved in OpenSearch 2.10, _efficient filtering_ filters *during* the nearest-neighbor search. In OpenSearch 2.14, we enhanced the filtering logic to further reduce query latencies for filtered vector search by approximately 30% at P99, using the Faiss engine. ## Indexing -So far, we’ve been discussing improvements to search latency and memory efficiency. But we have also improved indexing performance. Many workloads, notably log analytics workloads, spend most of their computing resources on the indexing phase. For these workloads, accelerating indexing performance can significantly reduce costs. In OpenSearch 2.7, we introduced segment replication, which improves performance up to 40% compared to document replication by eliminating redundant indexing. OpenSearch 2.11 introduced additional improvements to segment replication, allowing it to use a remote object store for improved durability and scalability. +So far, we’ve been discussing improvements to search latency and memory efficiency. But we have also improved indexing performance. Many workloads, notably log analytics workloads, spend most of their computing resources on the indexing phase. For these workloads, accelerating indexing performance can significantly reduce costs. In OpenSearch 2.7, we introduced segment replication, which improves performance by up to 40% compared to document replication by eliminating redundant indexing. OpenSearch 2.11 introduced additional improvements to segment replication, allowing it to use a remote object store for improved durability and scalability. ## Storage -OpenSearch 2.13 enhanced storage and retrieval efficiency by introducing more efficient compression codecs. These codecs reduce disk space usage without compromising performance. Additionally, OpenSearch 2.13 introduced enhanced data tiering capabilities to improve data lifecycle management. This allows you to reduce storage costs and improve performance on a wide range of hardware. +OpenSearch 2.13 enhances storage and retrieval efficiency by introducing more efficient compression codecs. These codecs reduce disk space usage without compromising performance. Additionally, OpenSearch 2.13 introduced enhanced data tiering capabilities to improve data lifecycle management. This allows you to reduce storage costs and improve performance on a wide range of hardware. ## Roadmap for 2024 -Here are some improvements we have on the roadmap for the rest of 2024. +Here are some planned improvements that we have on the roadmap for the rest of 2024. ### Core search engine -OpenSearch continues to innovate on the core search engine. Our full plans are available in the open on [GitHub](https://github.com/orgs/opensearch-project/projects/153/views/1). +OpenSearch continues to improve the core search engine. Our full plans are available on [GitHub](https://github.com/orgs/opensearch-project/projects/153/views/1). We are planning the following enhancements: -- Supporting fine-grained customer monitoring of query execution, allowing you to identify problematic queries, diagnose query execution bottlenecks and tune query performance. +- Supporting fine-grained customer monitoring of query execution, allowing you to identify problematic queries, diagnose query execution bottlenecks, and tune query performance. - Optimizing common query execution infrastructure by developing multi-level caching (tiered caching); faster interconnections among components using an efficient binary format (Protobuf). - Improving query execution itself, using document reordering to optimize query throughput and storage efficiency (graph bipartite partitioning); analyzing and rewriting complex queries to make them run more efficiently; dynamic pruning; count-only caching. - Modularizing the query engine to make future improvements faster and easier. ### Hybrid search -Hybrid search combines lexical and semantic vector search to get the best of both worlds. Highly specific names--- like part numbers---are best found using lexical search, while broader queries are often best handled by semantic vector search. We have supported hybrid search since OpenSearch 2.10. +Hybrid search combines lexical and semantic vector search in order to get the best of both worlds. Highly specific names---like part numbers---are best found using lexical search, while broader queries are often best handled by semantic vector search. We have supported hybrid search since OpenSearch 2.10. We are planning to enhance hybrid search in the following ways: -- By executing the lexical and the vector searches in parallel, we can achieve up to a 25% latency improvement. -- We will support re-sorting of the hybrid query results. -- We will support additional algorithms for combining the query results. In particular, reciprocal rank fusion has shown good results in some applications. -- We will return the raw scores of the subqueries, which are useful for debugging and for relevance tuning. +- By executing lexical and vector searches in parallel, we can achieve a latency improvement of up to 25%. +- We will support resorting of hybrid query results. +- We will support additional algorithms for combining query results. In particular, reciprocal rank fusion has shown good results for some applications. +- We will return the raw scores of subqueries, which are useful for debugging and relevance tuning. ### Vector search -Semantic vector search is a powerful technique for finding semantic similiarities. However, it can be costly, because indexing vectors to make them efficient for search is time-consuming, and the vectors themselves can consume large amounts of memory. +Semantic vector search is a powerful technique for finding semantic similiarities. However, it can be costly because indexing vectors to make them efficient for search is time consuming, and the vectors themselves can consume large amounts of memory. We are currently working on the following vector search improvements: - Accelerating vector indexing by 80% and reducing the memory required for indexing. -- Reducing the amount of space needed to store vectors by compressing vector components from 32 bits to 16, 8, and even 1 bit. Our experiments show modest reductions in search quality (recall) with 16 and 8 bit components; we are still analyzing the 1-bit case (binary quantization). -- Reducing cost by employing disk-based ANN algorithms that use external storage (SSDs, S3, and so on). +- Reducing the amount of space needed to store vectors by compressing vector components from 32 bits to 16, 8, and even 1 bit. Our experiments show modest reductions in search quality (recall) with 16- and 8-bit components; we are still analyzing the 1-bit use case (binary quantization). +- Reducing costs by employing disk-based ANN algorithms that use external storage (SSDs, Amazon S3, and so on). Vector techniques have also been less flexible than lexical techniques in many ways, so we are: -- Improving search relevancy by supporting reranking on compressed vectors. -- Supporting ANN vector search for the multi-tenant case, where each tenant has its own subcollection. This is especially valuable for customers each of whose user organizations has a large vector collection. +- Improving search relevance by supporting reranking on compressed vectors. +- Supporting ANN vector search for multi-tenant use cases, where each tenant has its own subcollection. This is especially valuable for customers each of whose user organizations has a large vector collection. ### Performance benchmarking -We are planning to update [the Big5 workload](https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/big5) mappings and settings. In particular, we'll add support for dynamic templates, configurable merge policies, match-only text, and setting the default query field as `message`. +We are planning to update [the Big5 workload](https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/big5) mappings and settings. In particular, we'll add support for dynamic templates, configurable merge policies, match-only text, and setting the default query field to `message`. ## Conclusion @@ -135,20 +135,20 @@ While we're continuing to expand OpenSearch functionality, we are also investing The OpenSearch team at AWS works in collaboration with the larger OpenSearch community. Without its contributions to testing, feedback, and development, OpenSearch would not be where it is today. -Stay tuned to our blog and GitHub for continuous updates and insights into our progress and future plans. +Stay tuned to our blog and GitHub for further updates and insights into our progress and future plans. ## Appendix: Benchmark tests and results -This section presents details on the performance benchmarks we used and the results we obtained. The data was collected using OpenSearch Benchmark to run the Big5 workload against different distributions and configurations of OpenSearch. In particular: +This section provides detailed information about the performance benchmarks we used and the results we obtained. The data was collected by using OpenSearch Benchmark to run the Big5 workload against different OpenSearch distributions and configurations. In particular: - We used OpenSearch Benchmark---our standard benchmarking tool---for testing. Rally might produce different results. -- We ran each test against a single-node cluster to ensure better reproducibility and easier setup. -- The instance type was *c5.2xlarge*: 8 vCPU, 16 GB, an average instance type. Using oversized instances can conceal resource usage efficiency improvements. -- The Big5 index was comprised of a single shard, no replicas (run with `--workload-params="number_of_replicas:0"`). -- During the test, we ingested data using a single bulk indexing client to ensure that data is ingested in chronological order. -- We ran the tests in [benchmarking mode](https://opensearch.org/docs/latest/benchmark/user-guide/target-throughput/#benchmarking-mode) (`target-throughput` was disabled) so that the OpenSearch client sends requests to the OpenSearch cluster as quickly as possible. -- We configured the indexing Log Structured Merge (LSM) with LogByteSizeMergePolicy because this optimization was in place last year, prior to the blog post being published. -- The corpus size of the workload was 60 GB, 70M documents. The store size after ingestion was 15 GB for the primary shard. If overhead like doc values is eliminated, Resident Set Size (RSS) should be about 8 GB, which should match the JVM heap size for the instance. Having most data resident in memory should provide a good assessment of performance improvements. +- We ran each test against a single-node cluster to ensure better reproducibility and an easier setup. +- The instance type was *c5.2xlarge*---8 vCPU, 16 GB---an average instance type. Using oversized instances can conceal resource usage efficiency improvements. +- The Big5 index comprised a single shard with no replicas (run with `--workload-params="number_of_replicas:0"`). +- During the tests, we ingested data using a single bulk indexing client to ensure that data was ingested in chronological order. +- We ran the tests in [benchmarking mode](https://opensearch.org/docs/latest/benchmark/user-guide/target-throughput/#benchmarking-mode) (`target-throughput` was disabled) so that the OpenSearch client sent requests to the OpenSearch cluster as quickly as possible. +- We configured the indexing Log Structured Merge (LSM) with LogByteSizeMergePolicy because this optimization was in place last year, prior to the publishing of this blog post. +- The corpus size of the workload was 60 GB, 70M documents. The store size after ingestion was 15 GB for the primary shard. If overhead such as doc values is eliminated, Resident Set Size (RSS) should be about 8 GB, which should match the JVM heap size for the instance. Having most data resident in memory should provide a good assessment of performance improvements. The following table presents the latency comparison. @@ -158,4 +158,4 @@ The following table presents benchmarking results by query category. -If you want to run your own benchmarking tests, please share the results with us. As always, we welcome your feedback and contribution. +If you decide to run your own benchmarking tests, please feel free to share the results with us. As always, we welcome your feedback and contributions. From eec181fa95fa875acb56f6d4ec65ced90b98584a Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Fri, 7 Jun 2024 15:19:21 -0400 Subject: [PATCH 3/7] Apply suggestions from code review Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _posts/2024-06-07-opensearch-performance-2.14.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2024-06-07-opensearch-performance-2.14.md b/_posts/2024-06-07-opensearch-performance-2.14.md index 64f58c3f0d..d6f03b75fb 100644 --- a/_posts/2024-06-07-opensearch-performance-2.14.md +++ b/_posts/2024-06-07-opensearch-performance-2.14.md @@ -20,7 +20,7 @@ We evaluated performance improvements using the [OpenSearch Big5 workload](https ## Performance improvements through 2.14 -Since the inception of the OpenSearch Project, we have achieved significant speedups. +Since the inception of the OpenSearch Project, we have achieved significant increases in speed. The following graph shows the relative improvements by query category as the 90th percentile latencies, with a baseline of OpenSearch 1.0. Every category improved considerably, some dramatically. Full numbers are available in the data appendix. From a45892a7fa27e27f971b38addaf8b785936553ea Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Fri, 7 Jun 2024 15:20:23 -0400 Subject: [PATCH 4/7] Update _posts/2024-06-07-opensearch-performance-2.14.md Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _posts/2024-06-07-opensearch-performance-2.14.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2024-06-07-opensearch-performance-2.14.md b/_posts/2024-06-07-opensearch-performance-2.14.md index d6f03b75fb..3d736105f8 100644 --- a/_posts/2024-06-07-opensearch-performance-2.14.md +++ b/_posts/2024-06-07-opensearch-performance-2.14.md @@ -133,7 +133,7 @@ We are planning to update [the Big5 workload](https://github.com/opensearch-proj While we're continuing to expand OpenSearch functionality, we are also investing in improving the performance, efficiency, and robustness of the system for every workload. In particular, vector techniques are being widely adopted both for pure semantic search and for hybrid semantic and lexical search. -The OpenSearch team at AWS works in collaboration with the larger OpenSearch community. Without its contributions to testing, feedback, and development, OpenSearch would not be where it is today. +The OpenSearch team at AWS works in collaboration with the larger OpenSearch community. Without your contributions to testing, feedback, and development, OpenSearch would not be where it is today. Thank you. Stay tuned to our blog and GitHub for further updates and insights into our progress and future plans. From ba343885e4fd000cb8baec63dcd45a879e5a5d11 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Fri, 7 Jun 2024 16:19:28 -0400 Subject: [PATCH 5/7] Add tables and Ian's profile Signed-off-by: Fanit Kolchina --- _community_members/hoangia.md | 23 + .../2024-06-07-opensearch-performance-2.14.md | 514 +++++++++++++++++- assets/media/community/members/hoangia.jpg | Bin 0 -> 19239 bytes 3 files changed, 518 insertions(+), 19 deletions(-) create mode 100644 _community_members/hoangia.md create mode 100644 assets/media/community/members/hoangia.jpg diff --git a/_community_members/hoangia.md b/_community_members/hoangia.md new file mode 100644 index 0000000000..b11275dc7c --- /dev/null +++ b/_community_members/hoangia.md @@ -0,0 +1,23 @@ +--- +short_name: hoangia +name: Ian Hoang +photo: /assets/media/community/members/hoangia.jpg +title: 'OpenSearch Community Member: Ian Hoang' +primary_title: Ian Hoang +breadcrumbs: + icon: community + items: + - title: Community + url: /community/index.html + - title: Members + url: /community/members/index.html + - title: 'Ian Hoang's Profile' + url: '/community/members/ian-hoang.html' +github: IanHoang +job_title_and_company: 'Software engineer at AWS' +personas: + - author +permalink: '/community/members/ian-hoang.html' +--- +**Ian Hoang** is a software engineer at AWS working on OpenSearch. + \ No newline at end of file diff --git a/_posts/2024-06-07-opensearch-performance-2.14.md b/_posts/2024-06-07-opensearch-performance-2.14.md index 3d736105f8..5fb36755e7 100644 --- a/_posts/2024-06-07-opensearch-performance-2.14.md +++ b/_posts/2024-06-07-opensearch-performance-2.14.md @@ -5,6 +5,7 @@ authors: - sisurab - pallp - macrakis + - hoangia date: 2024-06-07 categories: - technical-posts @@ -18,15 +19,64 @@ OpenSearch covers a broad range of functionality for applications involving docu We evaluated performance improvements using the [OpenSearch Big5 workload](https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/big5), which covers the types of queries common in search and log analytics, including text queries, sorting, term aggregations, range queries, and date histograms. This provides an objective and easy-to-replicate benchmark for performance work. + + ## Performance improvements through 2.14 Since the inception of the OpenSearch Project, we have achieved significant increases in speed. -The following graph shows the relative improvements by query category as the 90th percentile latencies, with a baseline of OpenSearch 1.0. Every category improved considerably, some dramatically. Full numbers are available in the data appendix. +The following graph shows the relative improvements by query category as the 90th percentile latencies, with a baseline of OpenSearch 1.0. Every category improved considerably, some dramatically. For the full results, see the [Appendix](#appendix-benchmark-tests-and-results). The heavy green line summarizes the overall improvements as the geometric mean of the individual categories of improvement, showing continuous progress in performance. -OpenSearch performance improvements up to 2.14{:style="width: 100%; max-width: 750px; height: auto;"} +OpenSearch performance improvements up to 2.14{:style="width: 100%; max-width: 750px; height: auto;"} ## Queries @@ -40,7 +90,7 @@ However, many applications do not need this additional information---it's enough For these applications, OpenSearch 2.12 introduced the [`match_only_text`](https://opensearch.org/docs/latest/field-types/supported-field-types/match-only-text) field. This field type dramatically reduces the space needed for indexes and speeds up query execution because there is no complicated scoring for relevance ranking. At the same time, it supports all standard text query types, except for interval and span queries. -**Using `match_only_text`, text queries are 47% faster in OpenSearch 2.12 than in 2.11 and 57% faster than in OpenSearch 1.0**. +Using `match_only_text`, text queries are 47% faster in OpenSearch 2.12 than in 2.11 and 57% faster than in OpenSearch 1.0. ### Term aggregations @@ -48,15 +98,15 @@ Term aggregations are an important tool in data analytics because they allow you OpenSearch 2.13 speeds up term aggregations for global term aggregations in immutable collections such as log data. This is an important and common analytics use case. OpenSearch gains this efficiency by using the term frequencies that Lucene precalculates. -**Evaluating term aggregations on Big5 data shows speed improvements of a factor of 85 to 100**. +Evaluating term aggregations on Big5 data shows speed improvements of a factor of 85 to 100. ### Date histograms -Date histograms are OpenSearch’s way of grouping data by date. Almost every Dashboard or Discover visualization in OpenSearch Dashboards depends on this functionality. For example, you might want to aggregate the logs of every HTTP request sent to your site by week. **Date histogram optimizations in OpenSearch 2.12 provide speed improvements ranging from 10 to 50 times on the Big5 benchmark**, in cases where there are no sub-aggregations into range filters. +Date histograms are OpenSearch’s way of grouping data by date. Almost every Dashboard or Discover visualization in OpenSearch Dashboards depends on this functionality. For example, you might want to aggregate the logs of every HTTP request sent to your site by week. Date histogram optimizations in OpenSearch 2.12 provide speed improvements ranging from 10 to 50 times on the Big5 benchmark, in cases where there are no sub-aggregations into range filters. ### Multi-term queries and numeric fields -Multi-term queries are commonly used in analytics to simultaneously aggregate by many terms, often retrieving only the top *n* results. **OpenSearch 2.12 accelerates multi-term queries on keyword fields by over 40%** by taking advantage of Lucene's IndexOrDocValuesQuery. +Multi-term queries are commonly used in analytics to simultaneously aggregate by many terms, often retrieving only the top *n* results. OpenSearch 2.12 accelerates multi-term queries on keyword fields by over 40% by taking advantage of Lucene's IndexOrDocValuesQuery. In version 2.14, we also use the IndexOrDocValuesQuery to increase search speed on numeric, IP, Boolean, and date fields, even when the fields are not indexed. This means that you can save storage space by not creating indexes for less commonly used search fields. @@ -68,7 +118,7 @@ Document vectors can require a lot of storage space, especially when documents a ### Vector search cost reduction -**By reducing the vector elements from 32-bit to 16-bit (fp16) floating-point numbers, OpenSearch 2.13 reduces the amount of storage required by 45--50%**. Our experiments show that this has little or no effect on the quality of the results: Recall remains above 95%, and query latency is unaffected. +By reducing the vector elements from 32-bit to 16-bit (fp16) floating-point numbers, OpenSearch 2.13 reduces the amount of storage required by 45--50%. Our experiments show that this has little or no effect on the quality of the results: Recall remains above 95%, and query latency is unaffected. OpenSearch 2.14 also improves the memory usage of IVFPQ and HNSWPQ vector indexes by unifying the storage of index metadata. @@ -103,12 +153,12 @@ We are planning the following enhancements: Hybrid search combines lexical and semantic vector search in order to get the best of both worlds. Highly specific names---like part numbers---are best found using lexical search, while broader queries are often best handled by semantic vector search. We have supported hybrid search since OpenSearch 2.10. -We are planning to enhance hybrid search in the following ways: +We are planning the following hybrid search enhancements: - By executing lexical and vector searches in parallel, we can achieve a latency improvement of up to 25%. -- We will support resorting of hybrid query results. -- We will support additional algorithms for combining query results. In particular, reciprocal rank fusion has shown good results for some applications. -- We will return the raw scores of subqueries, which are useful for debugging and relevance tuning. +- Supporting resorting of hybrid query results. +- Supporting additional algorithms for combining query results. In particular, reciprocal rank fusion has shown good results for some applications. +- Returning the raw scores of subqueries, which are useful for debugging and relevance tuning. ### Vector search @@ -135,7 +185,7 @@ While we're continuing to expand OpenSearch functionality, we are also investing The OpenSearch team at AWS works in collaboration with the larger OpenSearch community. Without your contributions to testing, feedback, and development, OpenSearch would not be where it is today. Thank you. -Stay tuned to our blog and GitHub for further updates and insights into our progress and future plans. +Stay tuned to our [blog](https://opensearch.org/blog/) and [GitHub](https://github.com/orgs/opensearch-project/projects/153/views/1) for further updates and insights into our progress and future plans. ## Appendix: Benchmark tests and results @@ -152,10 +202,436 @@ This section provides detailed information about the performance benchmarks we u The following table presents the latency comparison. - - -The following table presents benchmarking results by query category. - - - -If you decide to run your own benchmarking tests, please feel free to share the results with us. As always, we welcome your feedback and contributions. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Query typeOS 1.0OS 2.7OS 2.11OS 2.12OS 2.13OS 2.14
Big 5 areas mean latency, msText query44.3437.0236.1219.4219.4120.01
Sorting65.0418.5810.216.225.555.53
Terms Aggregation311.78315.27316.32282.4136.2727.18
Range query4.064.524.323.813.443.41
Date histogram4812.365093.015057.62310.32332.41141.5
Relative latency, compared to OS 1.0 (geo mean)100%78%68%30%19%15%
Speedup factor, compared to OS 1.0 (geo mean)1.01.31.53.35.36.7
+ +The following table presents benchmarking results by query category. It shows the P90 Service Time (ms) values. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
BucketsQueryOS 1.0 OS 2.7OS 2.11.0OS 2.12.0OS 2.13.0OS 2.14
Text Queryingdefault2.792.652.412.52.342.27
scroll448.9228.42227.36222.1210.41217.82
query_string_on_message180.55173.6168.298.729.298.22
query_string_on_message_filtered174.25125.88146.62102.53110.49135.98
query_string_on_message_filtered_sorted_num238.14183.62180.46112.39120.41139.95
term0.811.060.910.960.880.83
Sorting desc_sort_timestamp13.09159.4528.3920.6518.7620.4483
asc_sort_timestamp993.8161.9478.9139.7836.8333.79
desc_sort_with_after_timestamp1123.65163.5328.9319.418.7822.492
asc_sort_with_after_timestamp1475.5142.6938.556.225.555.14
desc_sort_timestamp_can_match_shortcut15.4931.137.647.116.546.88
desc_sort_timestamp_no_can_match_shortcut15.2930.957.637.176.536.74441
asc_sort_timestamp_can_match_shortcut198.5932.4618.9312.649.458.91
asc_sort_timestamp_no_can_match_shortcut197.3632.518.7812.749.028.82
sort_keyword_can_match_shortcut181.182.592.372.462.182.13
sort_keyword_no_can_match_shortcut181.062.432.442.492.152.09
sort_numeric_desc36.1935.1523.66.045.835.78
sort_numeric_asc66.8737.7421.145.35.155.2
sort_numeric_desc_with_match1.231.010.990.920.850.8
sort_numeric_asc_with_match1.240.990.90.890.840.8
Terms aggregationmulti_terms_keyword0.921.111.050.910.89
keyword_terms2126.252117.222382.371906.7112.746.96
keyword_terms_low_cardinality2135.162121.882338.11893.910.815.25
composite_terms696.37668.09631.23572.77581.24551.62
composite_terms_keyword1012.96943.35900.66827.16861.81826.18
Range queriesrange203.29170.68189.77115.38115.2125.99
range_numeric0.810.960.870.90.760.75
keyword_in_range210.69179.18200.37123.59124.66134.06
range_field_conjunction_big_range_big_term_query0.921.130.991.010.910.82
range_field_disjunction_big_range_small_term_query0.821.081.0410.860.81
range_field_conjunction_small_range_small_term_query0.831.090.930.980.850.81
range_field_conjunction_small_range_big_term_query0.830.980.880.910.780.78
Date histogramdate_histogram_hourly_agg3618.53664.453785.488.59.973.39
date_histogram_minute_agg2854.992933.762961.692518.692635.16148.69
composite_date_histogram_daily3760.54016.423574.181.441.511.39
range_auto_date_histo5267.475960.216055.746784.276977.056129.29
range_auto_date_histo_with_metrics12612.7713314.8713637.2313759.5114662.2413208.98
+ +If you decide to run your own benchmarking tests, please feel free to share the results with us. As always, we welcome your feedback and contributions and we'd love to hear from you in the [OpenSearch Forum](https://forum.opensearch.org/). diff --git a/assets/media/community/members/hoangia.jpg b/assets/media/community/members/hoangia.jpg new file mode 100644 index 0000000000000000000000000000000000000000..e4715af09a5eb18b43ca53d7d723997ea25b8881 GIT binary patch literal 19239 zcmbsQbyQnV*guK}r)ZH-oZ^H4MS>QmxVyJF1d3Y=h2kwK9yGXXC=?Bp7Pk^4KyXTN zrxb_E;rqUK{nlOgkGsyfch;J{Cu5n8dp?0@tBmH;sTVnRY9LIPqUBBBQmh)Kw3 z$;n7b$rz|C=t>a+7og3(uzi=787$SH}(LH zrhamXIrv3AAfbCi&%nsd!^;Qahloo^N=eJ8JX2Lu*U;25HZe6bx3IKwbaHlab%VP5 z2LuKMzj_@K9TOY(?tMHg4W6EnnU$S`C@C!~uc)l5uKCo|+|t_Cj_T;`>mL{#8Xg(N zOwY_>=jIm{f2?n8Zf)=E?(LucIy=9(yt=-*#R1^`FP!`5{{s4d!$WnC2Nxe751;5i zJUF<)_Ysc@pMYJ2kXp%*$ljNRLo||@Rynn}vF8D&n9)z5gWnVh9T(&W_vwF-{s+RR<+SO|5qu>u0AEQKEg7J^1V%_} zs0#T1D=r4I7vYz{K<~BqN+Jok1$jdNAduUCS$sIe1Kd*Q6_dDzT2MY*E54GH-psVj z2qjfo1L6OQ{nem+MMQ?02&?-6{WqxpFR>JohX7cCq|1kkaAAD^9}z?j5ae|y=l)+L z>CNfHAP|%faQ`0<(+yRH@i4jf0|1m2O({}u!?di@qB7H9_nq+-5!_e#uNt7zC07+7 z1M`CVE0U@{hBN~VRcQhDBNMy-DG4#a-A0}QN-SFF3X@z`0<7FF-ZM7~x z{d!%u;^l!EZ^wnVKV&9}tb0rURL#HD^1I$q@{j$JEdMTf=Q*oWPWzp5+l)gTRPk>3 zdm;s~bmj6@#;HWEkc7>k|0E$WoJDU@5#>|ffgAD%yyKja(FO+E?|OLaoa+9~|0Tba zt+11qx3SDRlczq8-T53vJiMn7GDrF2AK;Nvc&B}8rqUI1OYZA}+(BJSmv)AL%x!QL z6rl#qilv=}pvnxI|BrNN-cuhcT?#S4x#u2^D)&8g?kQwsgfYNTb1SQ$BO#}ai+pfj z27vONsN!;!q-CZmskxP8bl=mt(ueMzX9OkxOLa6rkk;j%)!X-U-%fHXi6G&-k4~S4 z@IATet6-uQ1yT8VWCa;pg%97@q|sC?jhj)Wg7oUbv@S`_x6U9a~{+_VzR3nXA2d$0=B#k?NM^s-) zL;4`4C!?Bw_c58hzf1AD6pvD4yIJ;PV?0nM`E_se)jIm8@+7dd5WRl@4!ne4P-BhM zcl=@QHfa!(%}Z{^PnuT^6#Q-{%D;zqcGcWb7<#fJaoZ#An4iRDWs%cq%S-P}5*cjL z>WFVr&c(l9yAC|!`C;jLr$V5Iuj!r-_e3tc=KvfMc&|GkJ{-3) zG%cVx-B6K$p{yhVPn8FHFX{-bd(DDF?lXDZ#@&jDqP2BOA#w6qkTJ}53PnjS!e$b8 z3eDq&7N*OBEMq2d@3rZ_t_JSA^4C;-{0JdkhGzJW(X}t0FYR4^^!Mi&q9#j&#&?37 z@P@1oJA`7;UOleqM`m*B)xaMF&)35+(yoqa)`f$eluXy}?z}Fva0S=t72oF6>&t72 z$mt7&h1FW@Ca~78KP~f!?4^j7bI*gnWsnVGZBAN>50)~#oweIm7iG<{$pqj8B7|iUaP? zBA#e%q@!~7)%ZyU$q-teKfppKYD7|clr2HY^ql1;2&!qea&Dfu9}Jx1PXbzv6m=Y$ zR#%WcOumKKF2?miNoh{bF8_|%2oZ^%BydT-b)IQepgN?gl89luEg(_rLGTKuB|)Y^PU&O8 zVITwKu4+5sM@BS7$elUr+ST0FQ4_WivGMMOisekQiK*nJWh*ij7AOW_o^G;a`tYB2 zHoM#_8{a*@0nJ7(_s@MT9gIr^3ql8D@VlM6x94d~bqVmc*}7U6DaNr+2C z!{HRdOP8TY03>Ukd42LiGWf(U=b@tFuov?Cua9~uZJA3U8?e*2jJ*Oa(hn(cy@+Pu zBeikcrZhp)q1E`)sS6?4mran9ydMM_XVeipCl@up^2Q{>rm!tsFjrBQ8Xp)aenKxQ z&JGI-F5&JE;&)n@>ureon91gyiWzfXwLR;Vb;4xQ;t1Dv=badj2%e z`3sQoR^mu;G%%2}7Xt=}8NG&mNK*uYIF=k4`0J`d)(eGdT_1998xXyjmB2RNP|)o^ zx!Zro6jcRpqJB0SFcdO0v$4YDDqb!0(V-|VN9tT$%hL12?uplr?zN<~nRYFij0uL& z%52+@@Gy>F{Lj+Q$Mvn+*1>9V2jfc@b@L+twiELJ?pGDBf%e}d5>@RV>+GD&)xMGX zyu#d&B#~MWGMD=7T8gKO@<*-6Z28U4FE2Q;HC82bJB`~};*5DPtN!_WLW{xxLHm-% zZbbxWhI?*J|2KVt#1tve_o}N35M-#B$}_^zEQREhp@r}HJndG23)e*GOd$X^_uO$q zKjy9^c`pe$oUvHG?@hRkHn&RSftWTTIZ$X{fc^Zw84om~`Ce8Qof^-LK}tm`RTbMX zK;V7H2tXL^@|0q%9Qo@<13(LLVx~mQQ1mt1->#6Y*TDmwFP9edN_STK*+dU$5GM{KD1SIKXzJ}ijMi51I71SCM}<|@(NfPgXk*X3a+PC*h_xj; zBL`a=;4g_ilMil`HItt9;(@>98|(n-IX{gbD(5($3wb?OAD{eO*OM{1ctS>GRcA0+ z^Q$uO3U9IbP(TRz1aE*daFeEoO)E{GZb6@Cep@z>n@}NN-@9f2e9Gl{oQSA23NsWb zCu*93TTe4?Pi0s5fg`w!b4R^q4`p69+Ny7M{9GoRE4)5Xl`-9)pub^~15IVA2ipxgYxAzpYL+1af}N+FK+L7o3za3s z?n+~SZA*(?I+&K2YJ?T2YziTOR}B)=L$Cifi(MGs>I~aQCB5BV^1M2xrHJ3joJ^Vn{+0+ z816HDFZzFLO*?l@l1&M1cm<~(t#-=JNTvwR=1nSqIe-xEsuupu&d} zLj6|eTjq{_qgoc#;8SS^!E+$S-Re!7-V(h~Xu<6G>b0XzWK5R1@i)HXS9rT%#8q%8 z%f8^7r`c_+vXPoFshyX;`85h!VHF=QZ;T`pCr7Tdg>ZgDKkeNqLt3EDHY9CrNryx!rg@6x)}2F)(L{DK*7yJ;C`MDMJ*XFO7|xZ*ceJ@Ll4 zlBbnzL4LB0503XrY>D*s8sC%DRFxQL$n< zcQ&dLzgd+@+Yyi5FoOSM@)6xeU!vF(Gy^{4f;s~;iEDh(qvya*!#|^#G>z^4`ge)* zOWypnK0Af(7ykfp@@$BK`j0L)0WgKjta^{}&(hR&b#Pw%HJXFx6Kl0o^6A)rfFZ9X z{q`g4Q2n>*+qYkjO;sw&kM3>F?ELup_J1Pvfndn+Kbu2Gn)IJpkup0UPD>&*1yCLZvS-H?3y=94ZG#u zn8#d)TS874tU2El|9U4M<7>=Y9!2FFY|Yxn<;WnO?_arO!?s%GXtvpYgh3n2oDzo;Cqzv+R$KR>mQD?HP_4?bD^rk?cyDqB%rW z=)mX16o`z`6y@x}6Qp5#ESrHB&a0(l_kCG$*R2n^X~Ir+IF~h1UD|N7k|eg2?+^nh z7|A(?7paMOP-NVI1Ia0Y3E&{Kth5B~JqJTOY8YQhH#co_%#~Uk>43Qmg}-kF zUKekHoJV%n^JFP<)1)sdp<(VUZpHrjh;MLXB7E~Fu1Iwdo1}ckN0#*zX|{Z;G-r?v zW?2p@xn7mqa`DHV=ZitelOaVdVxVQQZQj{iDYdKcE?%uRo6^pqJZPNS>I^6IsE zreP|c){<$muWs7Ll}r5GxA*?jNgnd+Y9c==#m82XIzm#>VFla876rRvn+X9m>!M@& zieTlDIAM_!OWvS>k+LM$=MA&g=h*b^XcaZA#1un5tciU*JjSrQbTZcUaqj z)yz7T(TF>L*y6UkWUzGAxp$k!r{%;$8N6}u^FE!pUVSNC3}q+0yP69NIWTTv{>l<- z2Jg=_0^ti^EorZUGkE>g_FqT{T!^a*X;aHk&6r9DhPu8#%+b8XER3}oosg9!YoRMW z8mm1V^;V6yy(TzXAm92fizty!H8BD!r$!bc)L{QPh?*Hs32L?PN47ZbT-NX;lNV+xIb=!zJeA;x+R*X~k!+|)!FZMNiaXu==00Vsd^j`4 zWKr-jTAK$Te$FRdxHi*Nf`kYrh?9ZHy_>^O1i%=oxji)cA755Oj{rbx|>2VT&eG7?w2HAx=t{g?iMP9TE)gaGA-Mf;d*DN;@K42d_ND3P>jt!*q0R;@y18j&tVM@ zTW*~S)2sWIuxvXOxSrvuk&!WvdKryr`rza!$3JGyp49uw{&p4=Zb_~prz7{mt^SHV zYg6ZANN?vU(WOTNRa5GpN!^ztJSI_MrjJ`Rg-9Pt233F1+z)ICaO7G%{>qk1=4{XF zM$igNf91aGE56d9;G36f%Y}7!z&mEUd{w8NDDo^OUe6;?s<=n}h)VrOyUE)!cNMwb z_1fo_dy|bY>NC_|j^*$SXV10gOk<80>-AQPW!@+Qyc3n`;_@F;^uBIX+QLEak2xdZ zkye+BKGWmo^x{Zr0%$Mq^;z+TlczVs6{JIQd)jo|x=ocH7iY4!@G2Vd+b0rLr-xdja)Yok~Dyl0hlP)(<839AH{Q%=(5j6c97{4 z3QG~@MEXBGpb}vd-fk)VY^JCL)PllxQa|~sIA!bn9v=OVYAtYax+=0|+urIbUp)I71h0sQ`A!N2JH>oAtKkv+# zlQAyFBGYT3imbAQD+cj?(^!S;Y$(Prs(%LTK&IaQopo3q9ql(Ye7b}5s@|oO@eP|M{1_kGOL&qL-ZH8LlGAm! zvS~JYa;3RxqDLoDYQ8Y~hx`G~kXBenLX?#F(?JHY==(RL5Oy7v_j1BPOxKn!|5vtn zAy;UbtV?sM%114a-p*H2MP*|Hk&?{5u+$X3VTQ86r`KGRRwWQrvsdh9*bp}^4bvAgZbrxubpisb z7|$u!>(oB;7T25v_`KduE1^J+r##1%nW+tu^T%Z!op!C{DSCGutM!ET9{~8ePG1<;?>N#Wc+Iq@e-$tr5((u|(ZN&?|6uQ`s``fU?5g9JM7d1`Fje(ZA$9yS}>y{i~nLFuvM4KNo4zDKM~LkJ{8ui&+Mxj_FS&+CJ7xpbeIY zI^CerOVzE90x(?7(Kh7XF!h6vb1*mK;n4VbY_{n+`>@t@B=NZZ`rn$SFt zAQl{{5quhXV6xp|liv+XdUh%Ey6$;s0}~=)r9Kp^y&<7p@%k?~piD4N-shl1T^S z5T%lnjAs7_aQo#x-c;68w|7KVLmHXxwKiAD7{0Fl&h|`qgjJw(!U0;BJV)4pdL!xU zHjm(tI~`EIb=W9(nEo;fA;jVcLL(i%j=>N!{{?VX2Z~8~8T{bQLnDqwl z^ohyDYMmBR8m{*PThd-3 zpISH#qg7xK&HQI#JgX(eD3oqRtunHW_LHJbKoyjBTEyGy?@K`=Q4wTc1>wt#qCH1U zW{h8FHenD^C2fHtyceMcsi!u~vGM-{1Oy_V!K6$$_zZ2+8`3gA zO}^uv?4N<`xlR`=YNV}5@<(5Fnll8c#C7*k5TA?h&5__0vkZa@vCYIEqqPY++vSFE zzO+nPy%W6(mMFGPT{wJwn7VdrO8P!}2ixlH>DmPUhI(Hb{%+4?^>J@|;D;tT+L?^0 z32m>Qu4>1&A@QJB!;YG?PH;qRR8px!48^Rw_)=0%vB~3++8{bfnew?*w;YB3o#U!2 zaXP;s)z+f=!6kKW^s?D0IYvV{a_{7Mbm^<0C2eoH_{`1Z&!J(U#pfS9$_E&q6>snV zq8j-JI52CcXP)yUVeV*}=yzZ~x27OM!FzRU+G{Fe34B-gl zg~s91*N{>}4wf9!C75a(Ylg&wN-#lVRzG5 zjXmX4BO-v&6z=;zujW3lqVwwCKVXhpptHxmU|nL70x*2`kRg5`h>~Yb)n(cGrUIU3 zRl?1b$0hS7?w9!e3TjWDrQw$&127pFVV(Xobk4bAkWOU4hk=ma)xo+^ZPi1C~zx^wex~%jOjJ4W-5c9{(Ua*inoffup^xx*61gW9oiWZQ>}$ zYA}w;v90uyzsUla%%Mcaf#%A+-`u@Vx;kdHe!D5K7;^~En~Ky zrN~vYY z5j|i`7Uo8-o&DwzwG+7tKa#&N=BbFntZy}v>HGsI4M?47F5u;|dgkoQnkzO`CyYiw%7`T}PI6DPtJ zZo5j+;}-M!qC*09l@U`Ey|M`*aKlQ$v=HN51kG`(HQ?sPR6c_g#0krZyT&;tc*aj? z)Y%aCixb$dwHv1D$5_>GAUnpXi^F z6A{8QdQ!yLtP)Tc)$=3?Mdu}(nxao2Fc0kkxfGL^d8axPh8!ucOFEc9gHve#Ua`E{ zb}up&qz5@+u8d1M?SZAg=3O7!?9pbum}!BZwKUR8;@EfaghiJ|UKGAcxW+<%LH{89|o&CK5!gn5?unGE@ADC!$)-My2b@hoZSb_gi^^j=i`RT^{ z{=ck@w_@!PNy^UTPN;!XH+Kj+nwAD!e;XU$!!Ip zV=1q!yK#xOZLp|qagwgGp04U(Z~^`G{Y;zo;79trNRRH)=Z}GxmmeKlzP5gC((bfO zv#@ctrBM2q#l`-`06Y^;XZvEYd?2>Bq1n#a%nO$};;rno5#v-r;kP#>$R1+Df(v@x z2xim#B}W1NEc;I&iaoika@wixa1<6Ik5bP1` zG=zSC#MUoks^oiMd(fUkGF()>4}OoW(@i{hsm$(WavN%%72kWujb@3?FHvazh?KH< zQdKvY7I85!&N_>5F$jf`l{EF;O>Hw@)EGZmwCUO=f!dqO_8n-rJf7aMAODaWD-thm zeuU2`Kex}HVf_G2hXcV_+=KkAcYf+u(4y7O(|Lcy6%BK*95(vzSoFrh)4_o@PZz!OX*oT#XZLBxvFY$}8lL(G88)h5_$*9>ZS!;Y^S&^J96otvX83*>_#?NWautp`faIjle)_76{JYXHP;HhTU`OI% z#|owT;;w%1JniAo%LEd|W*oRaJ#K!tVJwj5i!QiDSzitcIGj#r(@Ip+)uN;YV9RKc zI%qs*Jr;|QX`$_h@;}Z*c8QSe_iIm8T;&_Cyuj)-d=rn?Ud(8}rLtWjJGol;srS3g zV~nOl0;KvZOtJpJN7mH3Dv~Iv#eFQZSw7%%0#}SD-p?B9^9|VkXAgY30*->G4)v&& zn8%T+!}X>g#HyUw!#wEf4*5E3^Pfb?$}EmoZ|mf=#dvx9&zm<4RP+mS#qb$yK$nJq z`aO%XRxYu`s!U~yhpT0-f&Nm53Y@q@wQ}sto`A?^DC?i{*dvQiVr2Ip;An z(F4*>JqKEvnwF6}{%WT>Q|*1A9w42bN6D#fwr1upkzZda)RF$7@p2wJ9>xLN>kimf zw)RM+%u<*25|^H;MoqfUqso6T%*TogB5m?UV3FlJND)K{FAqZbWiFbv`Q$0k0z+eJ zpA%yjI_sL{sq+gUk_W36qVTBZEC3ta={=p=ft!>GAO|>f{IN(*YmVI%SIE=jnx2~h zS=`q}X3AoAv+un~)?8{d+|+g;WD9Zr?f$kD`9g~IdLVURSH(Dmbe$`8NWN%Zy}Jik z{cEQN&10I8bD*B&nB$)9+jj!}5*0t*op9|P^7UaZ;Z5_qA37Qv@eLQP1w_dBVWj>V zM}`czAfB!=Wbw1TK0K8jo}xJA>zAj9{9hI!#yCW@BI|Gn76KUbg68jOZfB@N;^b6i z*GQC9j2{j{dZB1+=>&w#0}=n8qukd(>aj~oEEwe{gH?YYsBw;D42{=Ew>eO3yOj7Jl+-d^bzo$zeAC+cRFNp^5Bkw38mSMxcm;m_n{d-B@@|BXO^!mjupdPj_)$A{ zXfm7@b*CQ#xk(bWH&rm~`D-YSA^BCf5>jQ9M!?~!oS;ek{a}(XGxeVBy6Ryx1m76F#ml0r zN}KQx5ZWcW)h7Rx9BRELo~I9HkNkcXsiCL(2v@leNcDy>zKkmihau3T+!e)2kI-(iVv0aLya_nt+&0g`GhsR zzh!1ed&c>unuIZ-`23?lX3Q!`a;gc^D_xYCUEZX@MBD2FEw8DgbBb-!EG$?5ii>!$ zry4Qn%8Q$bs?>>Kb}3#p?+WB_5ViRA8c38CZH91Gu!_kaf=mup?I1OZre&CChLv7~ z;4ASX97Imc@F1 z@$&Y(c8FU^ykLIVetvf5<)Ek=y~#`H=8PvAAju%k5v^;A);>}sO#+oo$FG+@OsX-H2RW)^8x+`KF4sKy?w}LAN!eptx3l3ia*ZlQtnnZ5mx$fZ4+fJE36jJCj zd!8*d`%4jBYjWPqC5nuu={oaHT7T-yYlB>zqt#FJSwy0tgUC0gW+(^W{xKBo?Bx?xvDW>cmxtrNpn10 zsmU_{Nvb8gCQ6erktC0mE>Z@2aUsIprBX7s3+4?q9MjjZFXB_W%0CzO(nnh<&e}S^ zL9@Ov?IRhYVs-bV9+!gYmgIJ11-^=Ej}4h|R(oCjpyHxJ6t^IQrrc&x%pRreQIy1A7eAI~RV9jC zBldS*lAJd{ZlK>$Q~IWU9MCi}nAb7X;?~^PQRQK~xQjF(d9^x|b||Mb*)L%O$&PSe zDC6X*2`j@_J5+O@@dH;qp2YvmS5dt@%k)K$-y)9C4QJe)8fuHGir>zrJkr!ekh3QN z$hqKTj*a+ptU5* zmX~{rjcuo<^|fHXHZT~zSMXNZ{R!>DTh~{G25qD<)S3E$Uh2CZ58Ox8RP|l)R-dc9 zjNZ=ZrL2~!S*k7jX`=L56S({#yUy^qa5p0Lc0FZvsA)H~&ez6zkSN1_eMMv!hV#eY z>NJ6*`=|fsFZ5B{IyYGa6%L`94sVeGfZr{jrtpJkS)m$b&!ID;oO^m?`tHUUI8Ov? z*>sICbOWI%bjWA`O&$=ICkN=bGtjgHANxR;9eK+E#xEn$Rx7PIg8-n`{twARB7~4v zzXZxlojGI@pP!)&N@+7=l>aNX=2Yh>Xr33k!R}Zn=e2s1b7m2WaP4|U_xLHs72!dn zyP`X9DrGU~enOQSjc#Vi_{oJ@7d5Zfn z1CLCzLrC5T7$rq{Qjb-(sL%|vA$5)|eE0{-Q-kn#>Y@I>kc#KYfbMUlnm!vjP8+*Y z^=WI^x3R^)WaVBK^#5%M^3n}Se_372)MLf>UYsjuzY^gU%4o)H7Msc-8AVCK|3o~2 z`r|3UE!!V$p7_3f>CX=sAZ=VdOD{I>;Xnvu7nFTbEyS3I4+f*egLzi>k9{eLcv2)V z>!40GWY~{OK3HQj(e;xL*HGxP^ed27PRs}f@G0j>rvNFyvPtylP%oHQ|9t3etEXU? z`^NTD2wa`=yJhED2Q2dIBuaDA?B`(Nx&!qPHFNeQD5^Nll=B>w=J{?}MC0YAI#j(k#y=hSn7 zc)yzWGY#2{h6HaZu_)8=)qK^UaJ}zjue^13=h_ne4h}UG-#&ID3^|T{H2+)L**nRI z*7%|u33-V$jd7R*1mIT0q)FF*z!l(@NeL(w=cd@eW@}Wsq)g-BoL9QiPU#{H85tF| zuVT$pH6X+FO0uVBq(oMLx~7bB#P0i~)#iob4OmrwbhIMyP2V>45Tf90r_Jz~+%W$) z>itE{oXlq*) z)P_7$)wM8p%cZS>XEhr46Bix3O#)`Qb)HM^lJSi?X@W8DHiw)NZFxfyFv;c*l=6r6 z6B^NtVukUv&vbRY{V&PY{1Mqny2AA!>M2%TrfQ7}L=Z4Lrvi>9s1aSBanf&yy;F_XqtZP#?IWQHX71^%h+IqxLCi(2D#vqKqp38s!9jj<-^OJ4a82{1{3jFcg$(J-DL6-v1cLeUM#A= zwighsS0n-|ME^dktkLJxaBwkoIf%rht#$`|T27JJt_2A!Bb;KI5SjQF6$Jf1K;K3x zk_a^EwXa7#1tH{;;bNDprWc;`FK zN?2#V`7}I#nmLS!4O8JYtZz&o)o-r4b})SSM%9aVo3w68i}K3G@h0UhNw4V+m(c@D z8&PJUG|?oMj=|+lui%?Sp{|pBfKhhi9IsUUqFvK({pmV5t63ql&sqQhO(C{GFs^(< zj?~7Pkg3x*Seixh8O1-q>|3kaH3FHBiBG)Cnc~giyp1lEEF+<;eN2NXRJ#NZM-wL} zTMxD#>eJU8P^RA>fUS2BLZd`9bXA#rq^l_*aeVGJ(T13+ks=~Qlzl4NVhH%$F;au< zMl?aAy13&|S2$t~Zn8Nlp;RVjdTeMeaDO0+kzA=ce$(H>3o8|q!E5{I4rt%yb@tY4 ziT9{1|5ENx64|(U_Tsx7SkhoFkkk@GOBHeREp-@JUUHMhsV^q>^vc~j@=MHlt?xqu zlNc}i)%vIS$ra=Vz-`Y7y##6{$P9z$5Yj?Pk%n(b3+CM}Ur}KdwYk1rsH{spMZ7Nb zvn_4oZo62B;1PMpy4r-9IG*?`quI1g4W}KL?!pk#ToBFY1+_5e=0&Q_5SF>F&2l|W zd}&Nv)Bof2i_X+9!boIuM$%8R~2Q*u8l=Yeml`KQo|YN_mT+3t{Dlo zb*X<99MI)@b6)kil`z}lmH)Ij$Ijzu!;&+!9!yL_E#pIKf-5pm!YPBH-dq2JKgqrz zoCJ($5^JptEGfK(t6bk->;1IiUXkEiI`BVfeeuJozs!17hw6{3;(_4vs`p5(hbivm z<38?Ke&mnd$%qNOSI#W@z(!}MOgbDzoMfI)FHxOJ`?For%ffsPAiOiv;YXb1s?7L# zvje)d1?h3*(-?${#Sl+vfgOPhTH1?0jWp{orhb{x$lpY;kD7nm7aPe?z}|UB5gY0| z?WCi0HSp%v0u6E;;etOOIAt?Zu2MdFZo&ZcR0dm*o!Z zIOASebtI;M1^p6Xop`4ziXSY{Qg+5l!$lbSx@(}3%QCsvk!en~Fgvf?;OiPV^K5lW zYP+cf`JRT8eFHS9_|Bw6R9E4fRQJ9cUfNvqRe?v4-lH)RUC)Ign7Hbmc70WAK zLjF*EQMEc%2w`kP@3E{Q_$E22H#HZ}9W$6dPt6AuB}pJ7GZ_{J5vQ1iBcH&_$NHpY zDF;6$WiVMImtA-_X^age=~4<7S%Zc(cm0)OCw{VxcHz63ZRAJgLEMOb!p`WZZO3k$ zUJosl@xEXBReIP|fUkRFUjEieQ6a`SE(h?J%1VIqSjdvq*H66)aao6_Zv%{Pn?v7$W$P^m z$8`-+i@Q)(($*IjqHNdXQqZfquct@6TQ)W1FZD&mJ{LR~;2+dLa$&re{@NC@X>}4t zhWO~Ll4uMkO)41bB4(caoHM0}4kEg((flkO`gmQw1vGm1V1!wv%GgM`GtXRO zm14$8!3XD}mTr}_%Zw=kSjmNK$mNIrbi^lFNqA}1>G30{6i+-%R)O^Row<=(}5ozyawJ~%6Mpb_V18Fmv_OZ zVuf3O#dk28g*J2g8#z}8NX7zAMFrNcSd?&iym-Pz%H$SMyZg zxS_U3Fq3^z3iC!fgx|mAaBRgrxJ~qKe6G((=C|~z(zwBmBiqCc?A#mII@kNb-f3#W z(AP8Rzb(Dj4x?M)EC)fW+Ba7Wd@#`viqgjQE>h(utXZBU)rPcM6lyX6g>nzQbnI@j zH{rRfvL;^iViEOmT`ki6sf#_wNCSvR5r-|)|JKNIwY>LTd|&aVFtS|D(A+KC0XlBe z7?M?CmSTtxk0Kfqm0|NjnxIUouL@|b00l&P^3Ob z%}usAzfBODJ^4{C2IAN&Wmibl`AHX%y1QBJ+d=CvZtl)vT4=t>5{?>(?oVo(;7j%xl zN**nLS+6J(=%gZbQ~%v$^@2y_H}3rf*-wmA_ey44`)5j96Glk?;ut1%UXqzY!o*VJ zCe5Af9#@(C{%=JM{~@s#<%H>CMf*R1r2!l<8^x%n`b-bBooxWu>tl36o40fIFI3O- zUmTd$Gl}FG#F%1ynuf18$w(&sEM|2d6Lj%~2qRGfkqS>}yT?g%L0WldJb9hoD8?s27LYb*rCWuI12SO}k%GlQha* zujl)3NUcrPH*Zc?2o0(d>wjaEd>nXmZPr}n_vLj#5GT#VU5Bf;bD)Q>x3^qg;|+PC zz+eR4zOZjHrs25A1nc^#3yzlLeop$#s3|9aNoD~O$S}f%IJtW{( ziA)}<48bS?cj!VS8huAZ$9BwZEl9YO$_)1y`p}t^XZ3wPVnPzl-?>fZ66K; zj%1`R9vJpVwpQCmJiWX%&%d|+SS@Xp>Dnx{e8TmWspsEqP&49l&Z%3C+?UsqE8D!*dgdO75{4O^6Gu_tmdL@75H`Vg*^b$kCg{ z^39RQIVNH^#SKad(@QNOnrC2E^B!*Q$@6rhZ(2A;s^*yb^{!@fm~%R4eymH9qU8UzVlV2 zvR3p(#j00p7%i$kFV}UG)^L=cMI7wFQ)i6vl78MEqH-v&;_Dl62K>z)Y`M3w_Rr~( z)pS#wZ!~E8nMzeh9d9yz@PhV%G1>7EJ(b3fkg|*b^&f7{3epwBP(l3@cZ)|&np#jW zI{~c=LjV3lD3$xKJjnr|vLcKDd;F6O`M!bY_*B@YKrV2r(h^eNgD>-;zv8zyhAE(Q z1H4}#F%2jm&QjHi4#WVj&2->JdKiD&1yHKJH#Nya<1xp*URbz)RnbrV)+kQnMAE<< z;%m^0r2Ha&Jp1E7W3Z&}6f0A)eN6-%!;dcZ{x^6SXlG zUKgG-v17?tkyTlRN=WhA8S_z3k#tS2zCYMEy_PjB+3jHL7iWhY$GpSyr<3j(nghoE zs15y~`&>@NBw~5um3o%gCb1)LHD0mT?DGDLW%c(}3!$2Fx<_K&R=TbEWCpe)EZYxW zO9FS;s%j*x0`m*w*QQBrdXuz!f@z1Szff%P{ywj-S*I&-iZr;<({xbicuLaq^B>^d zc_-ZmTTcfw401C#PP#`iIqkvb*+k3h`H{f~#_{`h&<=Q@OY83!exx4i%}uA=8~ed$ zVUcBR@lJUSJqCnfzdRr-YJV9f%Ri?cr-7?C>@@8LLTHXI_G|^a%Ht=!y}p~bddi;| zJ)$00!bumt5q?dSVD!zoLb|QP(`@L_-_}N(I|>odIvAaBet>O!K3li7;w?F<$7H|W!H{R&i;a-j+>jo&VI{t(%hjeL)v_Eb zxZfQ+y*8g70C79L@yII$`FH7_5bsM$gfx80NG)oY1#XZheN6&SNo=}`_gq`+P=7DG zF>1)$_!9b7=A2uPz{de&{7LTclTilKH;-NO`4EGLAH_Xf<@ianh~rQUdlscnof~yw z97E+KvEL7@gq326fOypO0NlDYj_o``0@v@hKqi_KMnY9zkxW}yxWVhnXJ5gRE#|8H zU$#GbE_kW80DMz_BbpOJM8ump`SCziR>QB&X*{LB*}t`@MVpm+7z4N0eu{JSFw*-G<7_(-+xt3I^v=xl zLw1E*GSVZ`vQ6Z*InFMgGR(;ng@cxljDtSKU{VzyXh7YHNHc?mN4a>Vj&wSrx$^7^ zwF*|he@l?ia74>>+ef`+#6ETeF205gnw?8kd)uYb3-9V%B}p%+F}qX|>)2gX(8t=I zv4^*pev3r`jPVrieZB`rR;(r}nB43k6-8PsoeLhYsO%YS(oEws)aH!$Ra8<=)ocS~ zD!PcX3uyI-Z%JS4ldc6$nG`N!0*2%oqU}!81OV5*(tKPz8KmKJ{O}6W!BTH&N9%(5o(K0RM zQe1^0im7sPK8aDS6jx0@;GA%o?xMJT9yp|1&1=lGjzP5?BIKWC%~!nBY_9a3 z9WG`;_LjQ3xW~$|w%$M;fGP!gL>exi3fg&AVLr)|07l_6*B@Vc$&6Or>c%)X?tQj8Yj_%aK&)J*9a`!>?9 z=b=5Z>s|1`s3BNwqFAC;L_vTQa4E7}fxtD%TCcR9G3;S5-&d!}dy>m2RVOu_E193# zm6@eDCal}r9OTw7Q>aU8e&(+#fGAYOMRRblp(NuJwgNlLrG*bX`c_H;?qqsG&2ujH z>7FWFb`&i}NeX$Ak3m;wAG;!%YaZ1sX>Olt{W+jTSrrDID&%CG)6K5}`fqK(PprArhchH+L9d1u%N%72&pRGv4BRy${CXiV5N~x$sWVa=L^7B@A4^c?&g-9$b zq{7B_pE;3so}#oX6oB6|EL{94 zasd^Z+U1bxClw3p_KG+p{uEsf%<5KkbQ?&|r9@;2*S%!e-p8s!#spx|oE98&Qkmw5 ziC1#^)~jI`*^))a1RkQP$!oVJHa|mAiPAxod{Xa}42BI(;8ba0Rw~)u@5M(X(IC$A z_*I#FnYfxEpYIN}0%^9a*=Kh21oWqHu@q9Y?SYjQWNuzD_|@3$;zi?V^u-~HFhetD zmY@;kQzYj!z`1c!mR8|g0tt}s7SUpzT14n=!c#9Obk_)=uf8fg+S`Q-lqD)DZo9qTHaf*TZ) zLo!Pf6m>aL4@$MFYLn`6#PfyBXb^23eQL7+2Ne-HsOFb1E%ajcadT5^R5v+2JJ!#J z@AYdPR^A;h>Edam3Ad`8p13?#K;!VPui^EuyYZEhu*i>egnb1~l%1J(aata$ZT|oX zCd^FMOn^^xt*+^SM0SOua z6k?jvV}VONnUR3?sK6xFgbglfs)fy1Sa%V{T8afwIHh5cLv4%_D$=buCl1*UCE8WYJot~8G?jG(;4icV-6buTM?l4u_pbDM{ORzMF8Dm%Cj3^Wc zZ~;>tg;HbY+yHtSv2!NW+B%ws-G6>hrxd{BZ%5hnIkEgNn%9gp&@8d?7-qAzxbdvW zgR2pW=%9!}3`xe&pc>AV?#g=<;-29+12t|NxftR^BBhep51lxsGq%hueEQbJ4wd=i zG{G?x(FN3<0UugMw3$X(hv!npP3HqZa8ET!-)8{xP?=vnO#q1eOAtt{=N$nmYo1aG z;8vfCE+rlva=fz0q*oA{z0xiM9QxNKZ6s|4c%4qr1xDD;PAix?wex=tKN^hP>JR)_ zthtNjXH(`N#$0}M&pse|aa^k#*6Sgr2yY4IR->3XPNlCc#|A-Be`s3{H`g+3=O-pc zKaC$@m;20r8aaZ;svE0@KkH30NoFg|jrr5S80k&G!L2T2Wyf(jyLH}3t3?^I#FO~d zRQ~{v*Q>p%P};Frq2!uqj~|U$AZ@0A05w2}N)QivLI6%`R;cryDS_GeZZ%C7Hri8p zC7af?9t1q?W9|-W4JmxB8rxwbC@NCydCm{Db;W3mNF=g}j^%BwRod2g@={Fx6?e>H zaBxSssRU|!RRNG>xKF+T9@N2m8nzr2tyYb2dekihZPh`f1Pd%~KXoVrzD{bEp?5XC ztg}W#ZT)K5wS`}V6&9e1EF@3{4-`FwUDJt@+9?@!mH~+LtAu3KcFF`O2S6%jrUb*K zRJm=39nD)EKMKjYk(yJJ(wGh{yS+y3!KUss$0r7WJkR2d{{S{tAo|L~^8&oxush(_ zTk#e~e-=m>@Q|Xf(;2QozV<7gZ{B4shLhX%q*01f(w8R!pkfRLz^_2~PiAD-Zm#f3 z#R))t*pKk80P)DLUieKCE{l06A3INiet=cMUP6)Rpq>HzDoun8=AaJ5=iaU{{nLFb zRV%v)&c;~OvBjR$B966bR1TFx1dnf8leHAd7=<{X22DD|q?1@zGrioqN8G1hO79?2 zcqW!4RRaQ{v4nB#6*8eFlmS*@L7tQdd{J#&)TD$`$DkCTas^b>9hFr9z$XTstOP9~ znoZrafmR?G$@Mi|1^~rdkAXr1&%9MEuz0=)-TSYQk@(kC;E)K;(m1Yn#*wMjB5ZT8 zR_2={J*)waeoD|g ztb1k2U^&f6;1#Yz%h!Y6elJ}4wDkD=`os7Yib5C TDYp{1Tn4B^9CfNQcR&BxzC$62 literal 0 HcmV?d00001 From 205feb5c96a97d892476ab82dd2051aab4a09763 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Fri, 7 Jun 2024 16:47:59 -0400 Subject: [PATCH 6/7] Last editorial comments Signed-off-by: Fanit Kolchina --- _community_members/ssingh.md | 2 +- _posts/2024-06-07-opensearch-performance-2.14.md | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/_community_members/ssingh.md b/_community_members/ssingh.md index ac9bb68807..5edb3936bb 100755 --- a/_community_members/ssingh.md +++ b/_community_members/ssingh.md @@ -28,5 +28,5 @@ conference_id: - "2023-north-america" --- -Saurabh, a Senior Software Engineer, is currently dedicated to the OpenSearch project at Amazon Web Services. His passion lies in finding solutions for intricate challenges within large-scale distributed systems, with a particular focus on leading the engineering efforts for the Security Analytics vertical. Saurabh actively contributes to the OpenSearch project, making contributions to its ongoing development and growth. +Saurabh is a Software Development Manager at AWS leading the core search, releases, and benchmarking areas of the OpenSearch Project. His passion lies in finding solutions for intricate challenges within large-scale distributed systems. diff --git a/_posts/2024-06-07-opensearch-performance-2.14.md b/_posts/2024-06-07-opensearch-performance-2.14.md index 5fb36755e7..5203dae5f8 100644 --- a/_posts/2024-06-07-opensearch-performance-2.14.md +++ b/_posts/2024-06-07-opensearch-performance-2.14.md @@ -102,7 +102,7 @@ Evaluating term aggregations on Big5 data shows speed improvements of a factor o ### Date histograms -Date histograms are OpenSearch’s way of grouping data by date. Almost every Dashboard or Discover visualization in OpenSearch Dashboards depends on this functionality. For example, you might want to aggregate the logs of every HTTP request sent to your site by week. Date histogram optimizations in OpenSearch 2.12 provide speed improvements ranging from 10 to 50 times on the Big5 benchmark, in cases where there are no sub-aggregations into range filters. +Date histograms are OpenSearch’s way of grouping data by date. Almost every Dashboard or Discover visualization in OpenSearch Dashboards depends on this functionality. For example, you might want to aggregate the logs of every HTTP request sent to your site by week. Date histogram optimizations in OpenSearch 2.12 provide speed improvements ranging from 10 to 50 times on the Big5 benchmark, in cases where there are no sub-aggregations in range filters. ### Multi-term queries and numeric fields @@ -197,8 +197,8 @@ This section provides detailed information about the performance benchmarks we u - The Big5 index comprised a single shard with no replicas (run with `--workload-params="number_of_replicas:0"`). - During the tests, we ingested data using a single bulk indexing client to ensure that data was ingested in chronological order. - We ran the tests in [benchmarking mode](https://opensearch.org/docs/latest/benchmark/user-guide/target-throughput/#benchmarking-mode) (`target-throughput` was disabled) so that the OpenSearch client sent requests to the OpenSearch cluster as quickly as possible. -- We configured the indexing Log Structured Merge (LSM) with LogByteSizeMergePolicy because this optimization was in place last year, prior to the publishing of this blog post. -- The corpus size of the workload was 60 GB, 70M documents. The store size after ingestion was 15 GB for the primary shard. If overhead such as doc values is eliminated, Resident Set Size (RSS) should be about 8 GB, which should match the JVM heap size for the instance. Having most data resident in memory should provide a good assessment of performance improvements. +- We configured the shard merge policy with LogByteSizeMergePolicy because this optimization was in place last year, prior to the publishing of this blog post. +- The corpus size of the workload was 60 GB, 70M documents. The store size after ingestion was 15 GB for the primary shard. If overhead such as doc values is eliminated, Resident Set Size (RSS) should be about 8 GB, which should match the JVM heap size for the instance. Having most data in-memory should provide an ideal performance assessment. The following table presents the latency comparison. From 1fc16d828a7b76ffa18cd82c46c4e382353f42d1 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Fri, 7 Jun 2024 17:16:17 -0400 Subject: [PATCH 7/7] Add new graph and small style improvements Signed-off-by: Fanit Kolchina --- .../2024-06-07-opensearch-performance-2.14.md | 16 +--------------- .../performance-graph.png | Bin 38923 -> 55841 bytes 2 files changed, 1 insertion(+), 15 deletions(-) diff --git a/_posts/2024-06-07-opensearch-performance-2.14.md b/_posts/2024-06-07-opensearch-performance-2.14.md index 5203dae5f8..bd0064b158 100644 --- a/_posts/2024-06-07-opensearch-performance-2.14.md +++ b/_posts/2024-06-07-opensearch-performance-2.14.md @@ -20,11 +20,6 @@ OpenSearch covers a broad range of functionality for applications involving docu We evaluated performance improvements using the [OpenSearch Big5 workload](https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/big5), which covers the types of queries common in search and log analytics, including text queries, sorting, term aggregations, range queries, and date histograms. This provides an objective and easy-to-replicate benchmark for performance work.