Merge branch 'hotfix-0.36.3'

Hotfix v0.36.3: Henkei: Avoid false linebreaks in plain text extraction

The previous method of plain text extraction from documents was causing
incorrect line breaks to be inserted into text.

The problem is that we're currently extracting plain text by first using
Apache Tika's HTML extraction and then using Nokogiri's HTML to text
method. That's because Tika's text extraction includes unwanted
artifacts, such as `[image: ...]` and `[bookmark: ...]`.
See [the TIKA-2755](https://jira.apache.org/jira/browse/TIKA-2755).

The solution to this is to use the Tika server's `/rmeta/text` endpoint
(as opposed to `/tika`) and then parse the JSON response. The plain text
content is in the "X-TIKA:content" field.

This field includes a bunch of line breaks at the beginning of the
document. These need to be stripped. In a future improvement, it would
be great to strip only those linebreaks that are not part of the
document.

Resolves [#342](https://github.com/OpenlyOne/openly/issues/342)

Author: FinnWoelm

CHANGELOG.md

@@ -1,5 +1,12 @@
 # CHANGELOG
 
+## v0.36.3 (Mar 4, 2019)
+
+**Fixes:**
+- Improve plain text extraction, so that we no longer have unexpected line
+  breaks in our diffs.
+  ([#342](https://github.com/OpenlyOne/openly/issues/342))
+
 ## v0.36.2 (Feb 18, 2019)
 
 **Fixes:**

lib/extensions/henkei/server.rb

@@ -26,14 +26,14 @@ class Server
     # Extract plain text from the provided file
     def self.extract_text(file)
       # HACK: extracting text via Apache Tika includes unwanted artefacts, such
-      # =>    as bookmarks and images. Use Nokogiri to get text from HTML
-      # =>    instead.
-      # put_file(
-      #   file: file,
-      #   path: CONTENT_PATH,
-      #   options: { 'Accept': 'text/plain' }
-      # )
-      Nokogiri::HTML(extract_html(file)).text.gsub(/\n\n\n+/, "\n\n")
+      # =>    as bookmarks and images. Use the /rmeta/text endpoint instead.
+      # =>    See TIKA-2755: https://jira.apache.org/jira/browse/TIKA-2755
+      response = put_file(
+        file: file,
+        path: '/rmeta/text'
+      )
+
+      JSON.parse(response).first['X-TIKA:content']&.strip.to_s
     end
 
     # Extract html from the provided file

spec/integrations/contribution_spec.rb

@@ -306,16 +306,12 @@
     end
 
     it 'gives creator of contribution edit access to files' do
-      expect(
-        api_connection.find_file!(
-          contribution.branch.root.remote_file_id,
-          fields: 'capabilities'
-        ).capabilities.can_edit
-      ).to be true
+      # Allow edit access to propagate
+      sleep 3 if VCR.current_cassette.recording?
 
       expect(
         api_connection.find_file!(
-          contribution.branch.files.without_root.first.remote_file_id,
+          contribution.branch.root.remote_file_id,
           fields: 'capabilities'
         ).capabilities.can_edit
       ).to be true

spec/regressions/extensions/henkei/server_spec.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+RSpec.describe Henkei::Server, type: :model, vcr: true do
+  subject(:server) { described_class }
+
+  describe '#extract_text' do
+    subject(:text) { server.extract_text(file) }
+
+    let(:file_name) { 'file-with-link.docx' }
+    let(:file)      { File.open(path_to_file_fixtures.join(file_name)) }
+    let(:path_to_file_fixtures) do
+      Rails.root.join('spec', 'support', 'fixtures', 'files')
+    end
+
+    it 'does not wrap links with new lines' do
+      is_expected.not_to include "\n[email protected]\n"
+    end
+
+    it 'trims newlines at document beginning and end' do
+      is_expected.not_to match(/^\n.*\n$/)
+    end
+
+    context 'when file has no text' do
+      let(:file_name) { 'image.png' }
+
+      it 'returns empty string' do
+        is_expected.not_to be nil
+        is_expected.to be_empty
+      end
+    end
+  end
+end