Skip to content

Ruby 2.7 compatibility fixes #93

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Sep 3, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
language: ruby
rvm:
- 2.3.0
- 2.7
addons:
code_climate:
repo_token: bcecbf1b229a2ddd666a2c3830f26a0113fd56ae1586d30d2d3fb1af837bf0e4
Expand Down
6 changes: 0 additions & 6 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,3 @@ source 'https://rubygems.org'

# Specify your gem's dependencies in csv-importer.gemspec
gemspec

gem 'rspec', '~> 3.3.0'
gem 'guard-rspec'
gem 'activemodel'
gem 'simplecov', require: nil
gem "codeclimate-test-reporter", require: nil
8 changes: 6 additions & 2 deletions csv-importer.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ Gem::Specification.new do |spec|

spec.add_dependency "virtus"

spec.add_development_dependency "bundler", "~> 1.8"
spec.add_development_dependency "rake", "~> 10.0"
spec.add_development_dependency "rspec", "~> 3.3.0"
spec.add_development_dependency "rake"
spec.add_development_dependency "guard-rspec"
spec.add_development_dependency "activemodel", "~> 5"
spec.add_development_dependency "simplecov"
spec.add_development_dependency "codeclimate-test-reporter"
end
27 changes: 23 additions & 4 deletions lib/csv_importer/csv_reader.rb
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,12 @@ def csv_rows
@csv_rows ||= begin
sane_content = sanitize_content(read_content)
separator = detect_separator(sane_content)
cells = CSV.parse(sane_content, col_sep: separator, quote_char: quote_char, skip_blanks: true, encoding: encoding)
sanitize_cells(cells)
cells = CSV.parse(
sane_content,
col_sep: separator, quote_char: quote_char, skip_blanks: true,
external_encoding: source_encoding
)
sanitize_cells(encode_cells(cells))
end
end

Expand Down Expand Up @@ -44,9 +48,8 @@ def read_content
end

def sanitize_content(csv_content)
internal_encoding = encoding.split(':').last
csv_content
.encode(Encoding.find(internal_encoding), {invalid: :replace, undef: :replace, replace: ''}) # Remove invalid byte sequences
.encode(Encoding.find(source_encoding), invalid: :replace, undef: :replace, replace: '') # Remove invalid byte sequences
.gsub(/\r\r?\n?/, "\n") # Replaces windows line separators with "\n"
end

Expand Down Expand Up @@ -75,5 +78,21 @@ def sanitize_cells(rows)
end
end
end

def encode_cells(rows)
rows.map do |cells|
cells.map do |cell|
cell ? cell.encode(target_encoding) : ""
end
end
end

def source_encoding
encoding.split(':').first || 'UTF-8'
end

def target_encoding
encoding.split(':').last || 'UTF-8'
end
end
end