incubator/elasticsearch Performance Suggestions

[rewt]

Is this a request for help?:
Is it possible to install plugins using this chart?

[rewt]

After running into issues with Paulo's kubernetes-elasticsearch-cluster and figuring out mlockall, I've carried lessons over to helm and happy to report things working well.

I'd also like to report some performance issues with the stock config.

Having elasticsearch service associated with client nodes reduces performance significantly if client nodes are not spec'd out with high CPU and RAM. Using stock version with client nodes, data loads perform extremely slow.

I'm not sure what the client nodes serve in this implementation and have instead pointed elasticsearch service at data nodes instead. Data loads perform much faster since those nodes are loaded with CPU and RAM. I do not see any benefit to running additional client nodes to handle requests at this time.

I've also worked out a solution for allowing mlock to run on kubernetes and have created repository https://github.com/rewt/elasticsearch-mlock

• Clone repo and build new image
• Load image to private repo
• Use private repo path in values.yaml
• update both master and data statefulset templates and client deployment as follows

insert

        securityContext:
          capabilities:
            add:
              - IPC_LOCK
              - SYS_RESOURCE 

directly above image path in elasticsearch container definition resulting in:

        - name: ES_JAVA_OPTS
          value: "-Djava.net.preferIPv4Stack=true -Xms{{ .Values.data.heapSize }} -Xmx{{ .Values.data.heapSize }}"
        {{- range $key, $value :=  .Values.cluster.env }}      
        - name: {{ $key }}
          value: {{ $value | quote }}
        {{- end }}
#begin new code
        securityContext:
          capabilities:
            add:
              - IPC_LOCK
              - SYS_RESOURCE
#end new code            
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"

• After updates, repackage chart and deploy.

Is there a way to build these options into the existing chart?

[desaintmartin]

Could the ulimit setting be put into the existing init container? This way, no need to have a separate image AND security is improved as only the init container needs the security authorization (well, at least in theory: kubernetes/kubernetes#65314).

About dedicated client nodes, it seems this is the official recommendation and allow to decouple requests from processing/storage in big clusters, but I'm not sure this is needed in small ones.

[desaintmartin]

After some thoughts, I would be interested into a PR (if official maintainers agree cc @simonswine @icereval @rendhalver) with a parameter defaulting to false doing the ulimit change. I maintain my own ES image with plugins to install plugins and it would help me reducing ad-hoc maintenance.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[stale]

This issue is being automatically closed due to inactivity.

[elieser1101]

After some thoughts, I would be interested into a PR (if official maintainers agree cc @simonswine @icereval @rendhalver) with a parameter defaulting to false doing the ulimit change. I maintain my own ES image with plugins to install plugins and it would help me reducing ad-hoc maintenance.

hi! @desaintmartin what was your workaround pull request idea to fix this the right way?